question

AI-curry avatar image
0 Likes"
AI-curry asked Ben Wilson edited

Why is it so hard to run a model after linking to a database?

I have linked the database well, but in the process of running the model, every time it read the data, the model will get stuck for a while, which is normal?

FlexSim 18.2.2
database
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

Ben Wilson avatar image
3 Likes"
Ben Wilson answered Ben Wilson edited

Hi @AI-curry,

When you read from a database, you are accessing data that is usually stored at rest on disk. Disk lookups are quite slow compared to RAM lookups - sometimes tens, hundreds, thousands, even hundreds of thousands of times slower, depending on your computer's hard drives and RAM.

This is usually fine if you are accessing your data only before a simulation run (to read data into your model) or after a simulation run (to write out results or other data). The bigger problems arise if you have to access your database during the simulation run. Even though you may have turned up the run speed of the simulation, you are now constrained not just by all the calculations that are going on to simulate your process, but to probably a greater extent you are constrained by the speed with which your system can talk to the database, which compared to running a simulation completely from RAM, could be quite slow (maybe even slower than real-time, though this depends on how frequently you're accessing the database, and the types of queries you're running).

Rather than accessing your database during the simulation, try importing the important data into global tables or other FlexSim data structures before running the model. Doing so puts the necessary data into RAM, where FlexSim can access the data quickly and natively. This could greatly speed up your simulation run (though it could also greatly increase the memory needs for running your simulation model - a RAM upgrade could be required, depending on your computer's specs and the size of the data necessary to run the simulation).

Not every simulation situation lends itself to this upfront data import, so you may be stuck. In that case, you can at least try to speed up your computer's I/O stack by upgrading your system to fast SSD storage. If you are currently running on a regular old spinning hard drive, this could make a large difference. There is also the chance that you are accessing a database on a second system or server, in which case there may also be network connectivity and latency issues. Finally, if your queries are complex or slow, you may find ways to restructure your data or rewrite your queries that makes it more efficient.

Again, it depends on your both your current system hardware configuration and how you've built your simulation model, but there are probably ways to optimize what is going on. If you have any questions or want to discuss your specifics in more details, please comment back.

Good luck!

· 2
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jason Lightfoot avatar image Jason Lightfoot ♦♦ commented ·

If you're accessing a database remotely over a slow network connection then you should try and use the new Database Connector with native connectivity. If you must use ODBC drivers over a slow network (and so using the long-standing dbopen() etc.. commands) then either get in touch with me or contact @phil.bobo and ask him to talk to me about a revised dll.

Also in line with Ben's post it may also be that Table Mode in the old connection method is slower than SQL mode (speculating here) - SQL mode will likely encourage you to import all the data before a run too.

1 Like 1 ·
AI-curry avatar image AI-curry commented ·

Thank you for your solution. I am not familiar with the mechanism of reading the database. Reading all the data before the model runs is really a good solution. My model does not need to read data dynamically while the model is running

0 Likes 0 ·