Solved

Shopify Sync Rate, how many per hour

  • 17 January 2024
  • 10 replies
  • 239 views
bryanb39
Rank
Userlevel 6
Badge +5

We currently import 2-3k orders per day from Shopify.  

They import at a rate of about 360/hour or 10 seconds per order.  

This usually keeps up, but with order updates and sales we can fall behind.  

What rates are you seeing on your system?  

Ideas are welcome, but trying to start with what are realistic expectations and what is possible.   

icon

Best answer by simonliang91 17 January 2024, 16:53

View original
https://www.fiverr.com/acudatica

10 replies

simonliang91
Rank
Userlevel 5
Badge +1

@bryanb39, if you don’t use parallel sync mode, there are only 2 threads to process the sync records. 

If you run the sync process in a schedule mode, I suggest you enable the parallel sync mode, it will use up to 8 threads to process the sync records, based on our testing, it can import 4000 - 6000 orders per hour. 

The steps to enable the parallel mode:


  1. You have a valid license in your Acumatica instance
  2. Add or modify these keys in Web.Config file 
    <add key="EnableAutoNumberingInSeparateConnection" value="True" />     
    <add key="ParallelProcessingDisabled" value="False" />

     

  3. Change the processor affinity for “w3wp.exe” in Task Manager, you may have multiple “w3wp.exe”, find the one with the same name as the name of the Application pool that uses in your site.
  4. Run the process data with “Process All” mode.


     
bryanb39
Rank
Userlevel 6
Badge +5

Thank you! Our Sys admin is working on the parallel processing settings.  Your 4-6K figure really helps us know what’s possible so even with that on we can have an idea if our hardware config and other settings are appropriate.  

https://www.fiverr.com/acudatica
bryanb39
Rank
Userlevel 6
Badge +5

Anything to be aware of for using this for 2023R1 and a processing node?  We are upgrading to 2023R2 very soon.  

https://www.fiverr.com/acudatica
simonliang91
Rank
Userlevel 5
Badge +1

@bryanb39 , the parallel mode is only active in the “Process All” mode in the “Process Data” screen, and we process the sync records by batch, each 50 records is a batch. 

simonliang91
Rank
Userlevel 5
Badge +1

@bryanb39 , The parallel sync mode works fine in all versions.

jamesh
Rank
Userlevel 6
Badge +6

@bryanb39, if you don’t use parallel sync mode, there are only 2 threads to process the sync records. 

If you run the sync process in a schedule mode, I suggest you enable the parallel sync mode, it will use up to 8 threads to process the sync records, based on our testing, it can import 4000 - 6000 orders per hour. 

The steps to enable the parallel mode:


  1. You have a valid license in your Acumatica instance
  2. Add or modify these keys in Web.Config file 
    <add key="EnableAutoNumberingInSeparateConnection" value="True" />     
    <add key="ParallelProcessingDisabled" value="False" />

     

  3. Change the processor affinity for “w3wp.exe” in Task Manager, you may have multiple “w3wp.exe”, find the one with the same name as the name of the Application pool that uses in your site.
  4. Run the process data with “Process All” mode.


     

Thanks, Simon!  We’ve been trying to figure this out for far too long! Haha

James Harris - ERP Guy & Acumatica MVP - If I answered your question, please mark my response as "Best Answer" :)
bryanb39
Rank
Userlevel 6
Badge +5

To make sure I understand this correctly, it looks like from the test I just ran that if there are fewer than 50 pending, it’ll still use a single thread, but if there’s 150 it’ll use 3 threads.

https://www.fiverr.com/acudatica
jamesh
Rank
Userlevel 6
Badge +6

@simonliang91 

Will this affect all parallel processing activities within the server?  If not, are there other keys we can add to increase performance from the application server and/or processing node via the web.config file or tweaks in IIS that we should be doing?

 

Thanks!

 

 

James Harris - ERP Guy & Acumatica MVP - If I answered your question, please mark my response as "Best Answer" :)
simonliang91
Rank
Userlevel 5
Badge +1

@jamesh , this is the article about the parallel processing: 

https://www.acumatica.com/blog/enabling-parallel-processing-on-acumatica-processing-screens/

In the connector, we use the fixed batch size 50.

jamesh
Rank
Userlevel 6
Badge +6

@jamesh , this is the article about the parallel processing: 

https://www.acumatica.com/blog/enabling-parallel-processing-on-acumatica-processing-screens/

In the connector, we use the fixed batch size 50.

This is great!  A game changer for sure! 
@simonliang91 - Can we get this to work for other screens, like reports with extensive post processing times after data retrieval? Like profitability reports and AR Aging, etc?

James Harris - ERP Guy & Acumatica MVP - If I answered your question, please mark my response as "Best Answer" :)

Reply


Terms & ConditionsCookie settings
About Acumatica ERP system
Acumatica Cloud ERP provides the best business management solution for transforming your company to thrive in the new digital economy. Built on a future-proof platform with open architecture for rapid integrations, scalability, and ease of use, Acumatica delivers unparalleled value to small and midmarket organizations. Connected Business. Delivered.
© 2008 — 2024  Acumatica, Inc. All rights reserved