c# - RavenDb importing data from different sources like xml, cvs -

we have system import alot of data. data can come different types of xml files , right mapped c# objects. want name (which unique, not id) update existing objects located name, or create new documents if no matching name.

how can accomplish in efficient way?

if didn't care checking existing, bulk insert.

besides that, need load each 1 name (because i'm assuming unique, said).

like this..

using (var documentsession = documentstore.opensession()) {     // doing import, allow lots of queries db.     documentsession.advanced.maxnumberofrequestspersession = int.maxvalue;      foreach (var foo in newfoostoimport())     {         var existingfoo = documentsession             .query<foo>()             .singleordefault(x => x.name == foo.name);          if (existingfoo == null)         {             // doesn't exist, save new data.             documentsession.store(foo);         }         else             // map required new data existing object.                    mapnewdatatoexistingobject(foo, existingfoo);              // save existing.             documentsessionstore(existingfoo);                    }     }      // commit these changes.     documentsession.savechanges(); } 

another alternative code approach reduce number of roundtrips db batching names. meaning, pass list of names where clause. maybe pass in 10 or 30 names. if exists, update records, else insert.

now if have large data set, might consider batching savechanges.

eg.

if (storecount % 1024 == 0) {     documentsession.savechanges(); } 

so every 1024 stores, savechanges. not sure if helps or not it's idea. (tip: if do, .. make sure have 1 more savechanges after loop u commit last set of data in loop.

hth.