question

Joe Hugan avatar image
1 Like"
Joe Hugan asked Joe Hugan commented

Large Data Set Handling

I have a large global table - 660000 rows and 22 columns that I have using bundled data. Trying to update this table with new data (it is an order file) is very time consuming and some times Flexsim will hang when trying to paste in a new data set. What is the preferred method for handling data sets of this size?

FlexSim 17.1.2
global tables
· 2
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Matt Long avatar image Matt Long commented ·

Are you doing a copy paste or an excel import?

0 Likes 0 ·
Joe Hugan avatar image Joe Hugan Matt Long commented ·

Matt,

The excel import too way too long. I ended up killing the process. I was able to get a paste to work when I started the project but updating it does not work with the paste. Keeps hanging.

0 Likes 0 ·

1 Answer

Matt Long avatar image
3 Likes"
Matt Long answered Joe Hugan commented

Looks like when we did an overhaul to our tabling system code was changed such that it's creating an undo record for each cell that is pasted (as opposed to a single undo record for the entire paste operation). I'll add this to the dev list so we can get it fixed.

· 3
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Joe Hugan avatar image Joe Hugan commented ·

Does that mean if I delete the table and recreate it, it should work faster? I need to get a solution for how to update this table until issue is corrected by dev.

0 Likes 0 ·
Matt Long avatar image Matt Long Joe Hugan commented ·

No, the issue is in the table interface. Have you tried using the Excel Import with the Data Distinction set to Values Only (very fast)? I imported 600,000 rows and 22 columns in under 20 seconds. In fact it took longer for Excel to open (about 14 seconds) than it did to do the actual import (3 or 4 seconds).

1 Like 1 ·
Joe Hugan avatar image Joe Hugan commented ·

That worked great! Thanks!

0 Likes 0 ·