Idea

Chris Smith suggested Feb 14 2017 at 4:51 PM Chris Smith commented Feb 17 2017 at 8:58 PM

Create a performance measure from statistic of every object in model, group, or class

The reports and statistics allows an export of statistics like the stats_input of every object in the model. Unless I am mistaken, in order to do this within an experiment I need to add a performance measure for each object to get the same functionality. If I have 1000 objects, then that would be tedious when done manually. Basically the idea is to have the functionality to select a measure that is a performance measure for every object in the model, every object in a group, or every object in a class (I imagine the universal selector would be used). Allowing custom code to create custom statistics that are applied to each selected object would be a bonus.

· 3

Phil BoBo ♦♦ commented · Feb 15 2017 at 6:41 PM

You can do this in the experimenter by adding a Dashboard that has those objects (easily added by class or group using the Dashboard Chart's Properties page) and then checking the "Save dashboard data for each replication" box on the Experiment Run tab.

You don't have to add a performance measure for each object to get that data.

2 ·

save-dashboard-data.png (185.5 KiB)

Chris Smith Phil BoBo ♦♦ commented · Feb 15 2017 at 6:51 PM

You're right that I can get the data that way, but, unless I am mistaken, I can't see a summary of all the replications for each scenario for each object. For instance, the only way to see the boxplot of the input value for each replications for processor 1 across the experiment scenarios, and the same for processor 2, and the same for processor 3, etc is to manually add a performance measure for each object. It would be nice if I could more easily add a performance measure for the input of every object (or a group of objects), so that I can get the summary charting that is done for performance measures. Does that make sense?

0 ·

Phil BoBo ♦♦ Chris Smith commented · Feb 16 2017 at 4:42 PM

Yes, but you can't do that with the Reports and Statistics functionality either.

The second half of your idea is a good suggestion, but it gets muddied by the first half that can already be done.

I've added your idea as a new item on the dev list. Thanks.

0 ·

Jordan Johnson commented Feb 16 2017 at 10:36 PM

This also brings up a very interesting effect called the Bonferroni Inequality.

When you experiment, and you run replications of a model, you get a confidence interval, not an exact value. You are saying that you are 95% confident that the mean is within some range.

But if you do that with many measurements, then you have to begin worrying about your model confidence, namely, what are the odds that all measured means are within their confidence intervals? If you measure 10 statistics, each to a 95% confidence interval, then there is only a 60% chance that all means are within their confidence intervals. If you are measuring 100 statistics, then there is a 0.5% chance that all of your means are actually within their confidence intervals. Each one might be very certain, but a few are likely outside that bounds.

So as far as decision making goes, try to focus on a few key measurements, rather than many.

· 1

______

Cookie preferences

Your privacy is important to us and so is an optimal experience. To help us customize information and build applications, we collect data about your use of this site.

May we collect and use your data?

Learn more about the Third Party Services we use and our Privacy Statement.

Strictly necessary – required for our site to work and to provide services to you

These cookies allow us to record your preferences or login information, respond to your requests or fulfill items in your shopping cart.

YES

Improve your experience – allows us to show you what is relevant to you

These cookies enable us to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we use to deliver information and experiences tailored to you. If you do not allow these cookies, some or all of these services may not be available for you.

YES

Customize your advertising – permits us to offer targeted advertising to you

These cookies collect data about you based on your activities and interests in order to show you relevant ads and to track effectiveness. By collecting this data, the ads you see will be more tailored to your interests. If you do not allow these cookies, you will experience less targeted advertising.

YES

Are you sure you want a less customized experience?

We can access your data only if you select "yes" for the categories on the previous screen. This lets us tailor our marketing so that it's more relevant for you. You can change your settings at any time by visiting our privacy statement

Your experience. Your choice.

We care about your privacy. The data we collect helps us understand how you use our products, what information you might be interested in, and what we can improve to make your engagement with Autodesk more rewarding.

May we collect and use your data to tailor your experience?

Explore the benefits of a customized experience by managing your privacy settings for this site or visit our Privacy Statement to learn more about your options.

Idea

Create a performance measure from statistic of every object in model, group, or class

1 Comment

Things to know…

Your Opinion Counts

IDEA DETAILS

Related Ideas

Cookie preferences

Strictly necessary – required for our site to work and to provide services to you

Improve your experience – allows us to show you what is relevant to you

Customize your advertising – permits us to offer targeted advertising to you

Strictly necessary – required for our site to work and to provide services to you

Improve your experience – allows us to show you what is relevant to you

Customize your advertising – permits us to offer targeted advertising to you

Are you sure you want a less customized experience?

Your experience. Your choice.

Idea

Create a performance measure from statistic of every object in model, group, or class

1 Comment

Things to know…

Your Opinion Counts

IDEA DETAILS

Related Ideas

Cookie preferences

Strictly necessary – required for our site to work and to provide services to you

Improve your experience – allows us to show you what is relevant to you

Customize your advertising – permits us to offer targeted advertising to you

THIRD PARTY SERVICES

Strictly necessary – required for our site to work and to provide services to you

Improve your experience – allows us to show you what is relevant to you

Customize your advertising – permits us to offer targeted advertising to you

Are you sure you want a less customized experience?

Your experience. Your choice.