Here is the binary equivalent of COPY FROM for Python 3: from io import BytesIO from struct import pack import psycopg2 # Two rows of data; “id” is not in the upstream data source # Columns: node, ts, val1, val2 data = [(23253, 342, -15.336734, 2494627.949375), (23256, 348, 43.23524, 2494827.949375)] conn = psycopg2.connect(“dbname=mydb user=postgres”) curs … Read more
I don’t really have experience with TVP yet, however there is an nice performance comparison chart vs. BULK INSERT in MSDN here. They say that BULK INSERT has higher startup cost, but is faster thereafter. In a remote client scenario they draw the line at around 1000 rows (for “simple” server logic). Judging from their … Read more
Alternatively, if you are wanting to disable all triggers, not just those on the USER table, you can use: SET session_replication_role = replica; This disables triggers for the current session. To re-enable for the same session: SET session_replication_role = DEFAULT; Source: http://koo.fi/blog/2013/01/08/disable-postgresql-triggers-temporarily/
DBCollection.insert accepts a parameter of type DBObject, List<DBObject> or an array of DBObjects for inserting multiple documents at once. You are passing in a string array. You must manually populate documents(DBObjects), insert them to a List<DBObject> or an array of DBObjects and eventually insert them. DBObject document1 = new BasicDBObject(); document1.put(“name”, “Kiran”); document1.put(“age”, 20); DBObject … Read more
There’s nothing wrong with your stored procedure code – the point is: the BULK INSERT command cannot accept a file name as a variable. This does work: BULK INSERT ZIPCodes FROM ‘e:\5-digit Commercial.csv’ WITH but this never works – within a stored proc or not: DECLARE @filename VARCHAR(255) SET @filename=”e:\5-digit Commercial.csv” BULK INSERT ZIPCodes FROM … Read more
Django 1.4 provides a bulk_create() method on the QuerySet object, see: https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.bulk_create https://docs.djangoproject.com/en/dev/releases/1.4/ https://code.djangoproject.com/ticket/7596
You need to use a ‘format file’ to implement a text qualifier for bulk insert. Essentially, you will need to teach the bulk insert that there’s potentially different delimiters in each field. Create a text file called “level_2.fmt” and save it. 11.0 2 1 SQLCHAR 0 8000 “\”,” 1 wkt SQL_Latin1_General_CP1_CI_AS 2 SQLCHAR 0 40 … Read more
Use the activerecord-import gem. Let us say you are reading a CSV file and generating a Product catalogue and you want to insert records in batches of 1000: batch,batch_size = [], 1_000 CSV.foreach(“/data/new_products.csv”, :headers => true) do |row| batch << Product.new(row) if batch.size >= batch_size Product.import batch batch = [] end end Product.import batch
Normally, each time db.insert() is used, SQLite creates a transaction (and resulting journal file in the filesystem), which slows things down. If you use db.beginTransaction() and db.endTransaction() SQLite creates only a single journal file on the filesystem and then commits all the inserts at the same time, dramatically speeding things up. Here is some pseudo … Read more
Add an id column to the csv file and leave it blank: id,Name,Address ,name1,addr test 1 ,name2,addr test 2 Remove KEEPIDENTITY keyword from query: BULK INSERT Employee FROM ‘path\tempFile.csv ‘ WITH (FIRSTROW = 2,FIELDTERMINATOR = ‘,’ , ROWTERMINATOR = ‘\n’); The id identity field will be auto-incremented. If you assign values to the id field … Read more