Ryusuke T avatar image
Ryusuke T asked Ryusuke T commented

About sqlite files


I would like to ask you about the sqlite file.

An sqlite file is created when performing optimization, but as shown in the image below, even if multiple optimization jobs are created, the sqlite file appears to be shared. Is this understanding correct?


If correct, are you referring to this sqlite file even if you have set different conditions in each job? I feel that the correct data may not be accumulated depending on the situation.What do you think?

FlexSim 23.0.11
2024022701.png (27.9 KiB)
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jordan Johnson avatar image
Jordan Johnson answered Ryusuke T commented

The SQLite file is shared across all jobs. Experiments and Optimizations all use the same results file. The database file stores the results of a simulation run. It is a cache of results. A job submits a series of questions, also called tasks: what if I run the model with these parameter values, replication number, stop and warmup time? The Experimenter responds by checking if those results are in the database file. If the results are available, the experimenter reports those results immediately without running a simulation. If the results are not yet available, the Experimenter will run a simulation, store the results, and then report the results back to the job.

When storing the results, it does not matter which job caused the results to be generated. All that matters is the configuration of the model (parameters, stop time, warmup time, and replication). If those inputs are all the same, the results of the run will be the same also.

So, let's say you have two optimizations with different objectives. How do these two optimizations work correctly, even while sharing the same file? The following steps describe the result:

  1. You can first run Optimization A with Objective A. Let's assume the database file is empty at this point.
  2. While trying to optimize Objective A, Optimization A will submit scenarios (sets of parameter values) it wants to try during the search.
  3. Each of those scenarios is submitted to the Experimenter. The Experimenter checks the results database file. Since that file is empty, all the scenarios will be simulated.
  4. Optimization A ends, and the best scenario for Objective A is shown on the chart.
  5. Now it is time to run Optimization B with Objective B.
  6. As Optimization B runs, it will also submit scenarios. If those scenarios have already been completed by Optimization A, Optimization B will see those results immediately, without needing to run simulations. This is because model behavior is not affected by an objective function. The objective function only helps the optimizer learn how to adjust parameters, and then test the effect of those model parameters.
  7. Optimization B may request many of the same scenarios as Optimization A. But it may also request new scenarios. All the new scenarios will be simulated. All scenarios, whether simulated or looked up, appear as a point on the chart for Optimization B.
  8. Optimization B ends, and the best scenario for Objective B is shown on the chart. This could be the same scenario that was best for Objective A, but it most likely won't be.

Essentially, the database file keeps you from re-simulating something that has already been simulated. A job doesn't know or care how long it takes to generate the results. Whether results are returned instantly (because they were cached in the database) or after a delay (because the results had to be calculated via simulation) is not important to the job.

Let me know if that makes sense, and if that answered your question.

· 1
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Ryusuke T avatar image Ryusuke T commented ·

@Jordan Johnson ,

Thank you for your detailed explanation.

This explanation helped my understanding very much.

0 Likes 0 ·
Jason Lightfoot avatar image
Jason Lightfoot answered Jason Lightfoot commented

The data should be keyed off the jobID so you can have as many jobs as you like without overlap.

· 2
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Ryusuke T avatar image Ryusuke T commented ·

If I want to manage both jobs separately, do I need to separate the files?

0 Likes 0 ·
Jason Lightfoot avatar image Jason Lightfoot ♦ Ryusuke T commented ·
I don't think so - what tasks are you hoping to do via the database? Have you inspected the schema to get an idea of how it's structured?
0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.