expected simulation run time for 155 million customer orders over 1 year

I have a data base of 155 million orders(rows) of over 6000 SKU's …now I need to model this data for given fulfillment centers.

Here my objective is to look for service level, inventory polices, demand distribution.

Here my question is, should I download data and import excel to flexsim or connect to data base read data over time.

Instead of reading entire one year data.

If I have such a huge data set in data base how I need to proceed in flexsim to make simulation more efficient.

Software Version:

FlexSim 23.0.0

· 7

Jason Lightfoot ♦♦ commented · Dec 08 2022 at 1:25 PM

Have you tried loading it into a table as bundle data? Also you could do some pre-processing into picks within orders so that you're not querying the large table on the fly (if queries are needed), and instead read order/pick data that has already been partitioned.

If you share the format of the table we can give some focussed suggestions.

0 ·

SudheerReddy Jason Lightfoot ♦♦ commented · Dec 08 2022 at 1:30 PM

@Jason Lightfoot

Currently data is there in client data base. we can download that as JSON but not sure on this.
When you say format of the table what exactly you mean.

0 ·

Jason Lightfoot ♦♦ SudheerReddy commented · Dec 08 2022 at 2:40 PM

The tables and their fields in the client database will tell us how it's structured and the type of queries that we could create to build a preprocessed data structure in FlexSim. The advantage of this is that when doing replications you will avoid each replication trying to get data from the database at the same time as it runs - placing undue load on the database. The disadvantage is that it will use some memory in FlexSim but with bundle data tables that might not be an issue.

0 ·

Allister Wilson commented · Dec 08 2022 at 1:26 PM

According to Microsoft (source), an Excel worksheet can only contain just over 1 million rows at maximum, so importing the entire dataset through Excel doesn't seem like it's even an option.

0 ·

SudheerReddy Allister Wilson commented · Dec 08 2022 at 1:31 PM

@Allister Wilson Yeah, connecting directly to data base and read data over time which is of one year of data. Is this some kind of solution possible?

0 ·

Allister Wilson SudheerReddy commented · Dec 08 2022 at 2:15 PM

Querying the database for 'chunks' of data sounds like a possible approach, yes.

I would definitely investigate the possibility of preprocessing the data that Jason raised though. Depending on what the data looks like, perhaps once you extract the bits you actually care about it might be a more manageable size.

0 ·

Jeanette F ♦♦ commented · Dec 13 2022 at 7:11 PM

Hi @SudheerReddy,

We haven't heard back from you. Were you able to solve your problem? If so, please add and accept an answer to let others know the solution. Or please respond to the previous comment so that we can continue to help you.

If we don't hear back in the next 3 business days, we'll assume you were able to solve your problem and we'll close this case in our tracker. You can always comment back at any time to reopen your question, or you can contact your local FlexSim distributor for phone or email help.

0 ·