Is it possible to use reinforcement learning to train AGVs?

Question

question

Vothan Salomão asked Sep 11, '23 Vothan Salomão commented Sep 27, '23

Is it possible to use reinforcement learning to train AGVs?

Is it possible to use reinforcement learning to train AGVs? I was thinking of using the Advanced AGV template in a network with multiple AGVs circulating, implementing the label of the Tokens for the current CP (Control Point), the last CP, and the AGV's destination in a global table, and creating a Parameter Table with this information for each AGV. This data would be used as observations for machine learning, and the actions would involve modifying the destinations of the AGVs or assigning each AGV to a specific destination. The goal would be to maximize the number of pallet outputs throughout the model, and I believe this approach could help optimize routes and reduce potential deadlocks.

I have read the available documentation on reinforcement learning

Software Version:

FlexSim 23.2.0

reinforcement learning optimize agvs

· 6

Kavika F ♦ commented · Sep 12, 2023 at 03:46 PM

Hey @Vothan Salomão, I haven't built any models like that, but by the way you've described the problem I think it's solvable. Is there a specific aspect of your issue you're trying to solve or figure out?

0 ·

Vothan Salomão Kavika F ♦ commented · Sep 21, 2023 at 02:29 AM

ReinforcementLearning_AGVs2.fsm

I would like to try building a model using reinforcement learning with AGVs, but I'm facing some difficulties.

First, it's related to the Process Flow and the Global Table I've constructed. I'm having trouble exactly capturing the CP and Last CP of the AGVs in the dynamic table I built throughout the route using the Process Flow and the CP and Last CP labels.

Another question concerns the AGVs' Destinations. In the global table, when referencing the Destination Label, the AGV doesn't actually update with the true destination.

The idea behind my model is to train an agent to make the best allocation decisions for the AGVs for Pallets periodically in the model, thus updating the Destination Label for empty AGVs or those on their way to pick up an item.

The agent's observations would be the columns CP, Last CP, and Destination. The actions would involve modifying the Destinations of AGVs that are not carrying items to pick up Pallets periodically. The reward function would aim to maximize the number of Pallets transported over time.

How could I update my Global Table more accurately by obtaining this information from each AGV and not from the Process Flow?

0 ·

reinforcementlearning-agvs2.fsm (14.4 MiB)

Jason Lightfoot ♦ commented · Sep 18, 2023 at 12:58 PM

Hi @Vothan Salomão, was David Seo's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

0 ·

Vothan Salomão Jason Lightfoot ♦ commented · Sep 21, 2023 at 03:13 AM

Hello @Jason Lightfoot I elaborated my questions in a better way after the initial inquiry, thank you

0 ·

Jason Lightfoot ♦ Vothan Salomão commented · Sep 27, 2023 at 04:57 PM

We've explained how to update the table in this post. Do you still need help with this question?

0 ·

Show more comments

Answer 1 · 2023-09-13T00:40:02Z

David Seo answered Sep 13, '23 Vothan Salomão commented Sep 21, '23

@Vothan Salomão

I take my client's case study as the example of using AI:RL looking for an optimized AGV route without dynamic traffic. It's not my project but the customer's project by themselves.

Yes. It's flexsim's python AI:RL feature.

So I think you can solve your issues like maxmize outputs through the model using AI:RL.

· 1

Vothan Salomão commented · Sep 21, 2023 at 02:42 AM

Hello @David Seo

Could you provide me with an example or guidance for my implementation? What parameters do you consider important for observations, actions, and rewards?

This is my model with the Global Table that I intend to use as an observation.

ReinforcementLearning_AGVs2.fsm

0 ·

reinforcementlearning-agvs2.fsm (14.4 MiB)

question