Mayhem Heroes Phase 2

What are your thoughts for having the most impact in OSS security for phase 2 of our Mayhem Heroes program?

One simple idea that I’m sure you have already considered is to just raise the minimum number of stars that a repo needs to be eligible, to capture some notion of “importance.”

It could be beneficial to offer customized project skeletons for the various languages supported by Mayhem that have the framework integrated into them. This would help push OSS developers to incorporate Mayhem and consider application security from the start of the project’s life, as opposed to having to convince them to incorporate it when the project is already large.

This is a fantastic idea! By language, and I guess by build system for c/c++?

When working on projects for Phase 1, I encountered situations where I was working with 20+ fuzz targets, and was even considering adding more than that. After speaking to Alex it sounded like having that many for a single project was fine, but would not work for every project.

I also encountered situations where I was waiting on workers/runners, not sure if this was from a shared pool, if each person was allotted a certain number of workers/runners, or if it was a combination of both.

As such I think for Phase 2 it would be beneficial to have:

  • More machines/workers/resources allocated to Mayhem Heroes
  • Some way to measure code coverage (and potentially a variable payout depending on size/coverage?)
  • An increased number of stars per repo like @/rnshah9 said could potentially capture some notion of “importance”, though larger repos do often depend on smaller repos/packages, so it really just depends.

It would also be nice if we could run fuzzing on github actions in parallel, since fuzzing actions would sometimes take 3+ hours (might go with what I mentioned above)

Definitely, perhaps even by the framework (e.g. Flask, Django for web development). I would definitely have a CMake, Make, Ninja, and Meson template for C/C++ as these were among the most common from my experience when integrating projects into Mayhem and the build process for the associated Dockerfiles was often times transportable from one project to another.

You are reading our minds here.

  • Mayhem itself can run targets in parallel, but our action implementation artificially makes it sequential. That’s something we will work on.
  • Code coverage is a WIP. We do binary-only now. Our current thought is to have a special build for compiled-in coverage (e.g., gcov), and we’re thinking about how to make this language-independent enough.
  • On stars, yeah, same thought. We’re also looking at a fork minimum.

Thanks for the input!