question

Hoang Nk avatar image
0 Likes"
Hoang Nk asked Kavika F commented

Question about File System API

Hi guys,

I'm having a question regarding the file system API. I'm trying to create the debugger where FlexSim will record the logging into an external log.txt file, I have done it by using the fileopen(), fpt(), etc.

However, now I want to improve it by reading the log.txt file size first, for example, if the file size is over 50MB then it will be renamed to log.001 and FlexSim will write to a new log.txt file, so in the long run the log files in archive will be "log.001", "log.002" and so on.

I have read this article: https://answers.flexsim.com/idea/47433/proper-file-system-api.html and understand that FlexSim can't read the file's size as well as delete or rename files. Are there any updates on this matter at the moment?

Regards,

Hoang Nguyen

FlexSim 23.0.4
event loglog filefile system api
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Kavika F avatar image
1 Like"
Kavika F answered Kavika F commented

Hey @Hoang Nk, we currently don't have a File System API for FlexSim. However, I think there's a way you could achieve a solution. You could use the Python Connector to write a python script which can do all the things you're asking (detecting file size, renaming files, etc). In the mean time, I'll add it to the dev list to discuss implementing a File System API.

· 3
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Hoang Nk avatar image Hoang Nk commented ·
Thank you for your suggestion, I'll try that. But just out of curiosity, do you happen to have any example model that connects to an external Python code? It would be great for me to look at a reference.
0 Likes 0 ·
Kavika F avatar image Kavika F ♦ Hoang Nk commented ·

There's a post I made a while back on how to use Python to generate data to populate a Global Table. It has a step-by-step guide on setting up the Python connection, your code, how to format it, and more. It contains useful links to the documentation that may help you set things up.

0 Likes 0 ·
Jordan Johnson avatar image
2 Likes"
Jordan Johnson answered Hoang Nk commented

Short Answer

No, we haven't changed the file API.

Long Answer

You may be able to do what you want with a different approach, but it depends on what you need the files for. My first guess at why you need the files split apart is that you want to search through the logs for certain information, but working with .txt files larger than 50 MB is difficult.

If that is the case, consider using a different approach for event logs. It's more complicated, but gives you a lot more flexibility in how you search. Here's a demo model:

EventLogDemo.fsm

The basic premise is that you can use a Database Connector to create and search the event log, which is stored as a SQLite database.

To use it, first update the Connection String in the EventLog database connection. I'd recommend running this script and using the result:

return modeldir() + "EventLog.db"; 
// On my PC, this returns D:/Downloads/ManualTesting/EventLog.db.

There is a User Event that fires at time zero. The user event connects to the database. For SQLite, this means creating the file if it doesn't exist already. The User Event also runs some preliminary queries to create the event log table. The PRAGMA statements are there to make logging faster.

While the model is running, the Queue and the Processor call a user command in their triggers. The user command adds a new entry to the database.

Once the model runs, events are added to the file. But how do you get them back out? You can use the Import tab on the EventLog database connection. If you click the Import All Now, button, you'll see the result of your query in the Global Table called Events.

The query is current as follows. I've added explanations so you can change the terms and click the Import All Now button again:

SELECT * FROM events   -- get all the columns from the events table
WHERE msg LIKE '%out%' -- only include rows that have "out" somewhere in the msg column
LIMIT 20,30            -- skip the first 20 matching rows, and then show the next 30

You could also just show events from a certain time range:

SELECT * FROM events
WHERE time >= 1000 AND time <= 2000

Pros and Cons

The benefit of this approach is that you can search your logs in a much more powerful way. You don't have to try to split your file just to be able to search it. However, instead of doing simple fpt() calls, instead you need to deal with a database and SQL. That can be challenging, especially if you aren't familiar with those topics already.

In addition, since I don't know your actual goals for the files, this may not be a solution at all. For example, if you are splitting the files to send them to someone else, then this approach won't help you. But maybe someone else will find this approach helpful.


eventlogdemo.fsm (28.5 KiB)
· 1
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Hoang Nk avatar image Hoang Nk commented ·
Hi,

Thanks for your input. I'm actually trying to split the log file because we have a big model and it will be running for a very long time, continuously, so the data size can become up to 500 MB and that is not good for the performance. Therefore we are trying to find a way to automatically split the file out. I will be looking at the external code option as Kavika suggested below.

Regards,

Hoang Nguyen

0 Likes 0 ·