Payment Matrix Clarification

Mayhem Heroes: Making OSS more secure
1

I would appreciate some clarification regarding the mayhem scoring system, especially in cases where multiple targets are involved. For example, if one target achieves >100 tests per second and another target has >100 test cases, while a third target finds a defect, could you kindly explain how the points are distributed in this situation? Is there a specific order or priority in which the points are awarded? Also, are the points cumulative or mutually exclusive? A concise and comprehensive explanation would be helpful in understanding the point allocation process for such scenarios.

Additionally, if a corpus is not used, mayhem will continue to use the previously generated test cases. Does this contribute to the total, or does the 100 test cases have to be generated within a singular run?

1 Like
3

The requirements are taken on aggregate. Meaning, if one target finds 100 test cases and another meets the speed requirement, you should receive credit for both.

Test cases can be cumulative, but in the tooling we use to score submissions, we are checking for generated test cases, so a seed corpus will not count towards the final number.

4

If I have two targets included in the repo that both find 50 test cases, would that count as 100 in aggregate?

5

Yes, that’s right. We count the target test cases on aggregate.