# mysql DJANGO_SETTINGS_MODULE =settings_mysql tests/manage.py bulk_it # default - sqlite DJANGO_SETTINGS_MODULE =settings tests/manage.py bulk_it The ( SOFTCORE) performance test looks promising, more than 70% less time (average): $ make testcmd SELECT all from database (based on the match_field parameter).bulk_update_or_create: 1 BIG_SELECT + 1 BIG_UPDATE + (lte_N) INSERT.update_or_create: (1 SELECT + 1 INSERT/UPDATE) * N.This package tries to tackle this introducing bulk_update_or_create to model QuerySet/Manager: ON DUPLICATE KEY UPDATE) look interesting (TODO on different package) but they will be retricted by bulk_create limitations => cannot use on models with multi-table inheritance bulk_update does some nice voodoo to update several records with the same UPDATE statement (using a huge WHERE condition together with CASE), but you need to be sure they all exist.bulk_create is really fast if you know all records are new (and you're not using multi-table inheritance).Searching online shows that this does indeed happen to quite a few people though it doesn't seem to be any good solution: If the number of records is big, the slowliness of QuerySet.update_or_create will stand out: it is very practical to use but it always does one SELECT and then one INSERT (if select didn't return anything) or UPDATE/. Everyone using Django ORM will eventually find himself doing batch update_or_create operations: ingest files from external sources, sync with external APIs, etc.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |