Don't do this #2

Simple iteration tip...

You are required to load "sh*t loads" of data from the back-end data storage.
And then this data needs to be iterated.
And data contains relation data which needs to be loaded.
And child data needs to be iterated.
And then both parent and child records must be combined into some output
...
And then you figure out it's extremely slow on large data.

So some pseudo code:
NOTE: if you can't read pseudo code, you're not the developer ... yet ;) #punIntended #comeBackWhenYouAreCapable

So I've seen this kind of code 1 too many times!

query = '...';  //just some query that loads 100000+ items from storage
result = dataStorage.loadData(query);
fileData = []; //target array we need later for the output
for (idx = 0; idx < result.size(); idx++) {
  item = result[idx];
  for (idxRelated = 0; idxRelated < item.related.size(); idxRelated++) {
     relatedPartialData = item.related[idxRelated];
     fullRelatedData = dataStorage.loadById(relatedPartialData.id);
     prepareDataForExcelFile(item, fullRelatedData, filaData);
  }
}
createExcelFromFileData(fileData);

instead do this:
more pseudo code:

fileData = []; //target array we need later for the output
query = '...';  //just some query that load 100000+ items from storage
result = dataStorage.loadData(query);

//use maps !!!

//map of items by item.id
itemsMap = {itemId => item};  

// map of related data by related.id
relatedMap = {relatedId => related}; 

//map of related.id(s) by item.id.   
//Note: have an array of related ids !!! 
itemRelatedMap = {itemId => [relatedId]}; 


//iterate items
for (idx = 0; idx < result.size(); idx++) {
  item = result[idx];
  itemsMap[item.id] = item;
  for (idxRelated = 0; idxRelated < item.related.size(); idxRelated++) {
    relatedPartialData = item.related[idxRelated];
    itemRelatedMap[item.id] => addToArray[relatedPartialData.id]};
    relatedMap[relatedPartialData.id] = null;
  }
}
//load and iterate related
relatedResult = dataStorage.loadById(relatedMap.getKeys());
for (idx = 0; idx < relatedResult.size(); idx++) {
  relatedItem = relatedResult[idx];
  relatedMap[relatedItem.id] = relatedItem;
}

//voila .. 
//with only 2 queries you have all the data in maps and ready for processing.

prepareDataForExcelFile(itemsMap, itemRelatedMap, relatedMap, fileData);
createExcelFromFileData(fileData);

Lesson #1: don't iterate and execute queries!
Lesson #2: if you think your result set will always be small, read the lesson #1 again.
Lesson #3: for sorting purposes use proper collections or implement custom sorting function ;)

Armando Ota @armando_ota

Don't do this #2

Comments 0 total