It has been about two months since I pushed out the previous update to Lighthouse, a code coverage plugin for IDA Pro. Overdue for an update, I spent the past several days adding new features to the plugin, addressing feedback, and fixing up bugs that have accumulated since the v0.6 release.

This post marks the release of Lighthouse v0.7 - Big ticket changes include Frida support, a right click context menu, improved usability for large IDB’s, C++ name demangling and a multitude of other tweaks + bugfixes.

Frida Support

In October @yrp604 submitted a pull request to Lighthouse with an instrumentation script capable of collecting Lighthouse compatible code coverage using the Frida DBI framework.

On Twitter he asked for more Frida-proficient eyes to review his contribution and @oleavr, the creator of Frida, stopped by to offer his personal expertise on the pull request. Can’t ask for better support than that!

Frida is scriptable instrumentation framework that is particularly well known for its support of mobile platforms such as iOS, Android, and QNX. This is a quality addition to Lighthouse’s existing support for DynamoRIO and Intel Pin.

Support for Frida will be marked as experimental while the community helps us shake out any early bugs.

Context Menu

The table in the Coverage Overview now comes with a right click context menu. I have populated the context menu with a few basic amenities to enable quick user interaction with the table entries.

Function prefixing was added to this menu as a request from the community. It operates in a similar fashion as the standalone IDA prefix plugin, but is only accessible through Lighthouse’s Coverage Overview table.

Please try out the context menu when you get a chance and let me know what other actions you might like to see added to it in the future. Community requests help me keep the pulse on what is important.

Large IDB Support

While using Lighthouse with some rather large IDB’s (250k+ functions) I encountered a few annoyances that degraded its usability at these extremes. This release promises improved support for big databases.

When loading coverage data from disk, Lighthouse builds a cache of the IDB metadata in memory so that it can operate in a performant & asynchronous manner when the user begins to explore their coverage.

Previously, the metadata cache would get rebuilt every time a selection of coverage files were loaded by the user. This was largely unnecessary since the only reason to refresh metadata is when function structure gets modified, or functions have been defined/undefined in the database.

Lighthouse will now only build the metadata cache the first time you load coverage files. If a refresh is ever needed, one can use the ‘Full refresh’ option from the new right click context menu in the Coverage Overview table.

Additionally, I have given the shell’s search command a bit of breathing room when operating on a large IDB. It will now wait for one second after your last keystroke before evaluating the search request.

Minor Tweaks

There have been many tweaks made throughout this release. The following is a selection of the minor changes that you might find interesting or valuable going forward.

C++ Demangling

The Coverage Overview now shows demangled C++ names. This is a no-brainer that should have been present from the first release. Thanks to Florian for bringing this to my attention.

Clear Aggregate

The special aggregate coverage set now displays a red ‘X’ in the coverage combobox. This can now be used to clear all loaded coverage from Lighthouse in two easy clicks.

Coverage Color Change

A few users using the default IDA theme raised the point that the light green coverage paint was throwing off their feng shui when paired up against the blue/red coverage table. The use of green was an artifact of an older theme.

The light green paint has now been switched to a light blue for users using a light disassembly theme.

Jump Case Sensitivity

A minor quality of life / usability tweak has been made to the shell’s jump functionality. It is now case insensitive for functions starting with the reserved sub_XXXXXXXX prefix.

Fuzzier Module Name Detection

When loading coverage data from disk, Lighthouse automatically extracts coverage from the log based on the name of the executable that was used to generate the IDA database.

In this release I have modified the coverage loader’s name matching ‘fuzziness’ to ignore file extensions as a fallback, focusing only on correlating module names between the log and the IDB. This frequently tripped up Lighthouse when loading coverage collected against Linux libraries (among other things).

In the future I plan to add a dialog for the user to manually select coverage from the log as a final fallback.

Bugfixes

With every release of Lighthouse I generally find and fix a number of bugs. This section highlights some of the important bugfixes that will be shipping with this release.

Propagating Function Renames

Previously, the metadata cache would not get updated when functions were renamed in the database. This issue has been on the backburner for awhile, but last week MJavad opened an issue for it which prompted me to finally fix it.

Incorrect Coverage Percentages

Lighthouse has been incorrectly computing the database-wide coverage percentages. It has also been incorrectly computing the coverage percentage when using the / (search) command on the composing shell.

The percentages in the locations indicated above should now be closer to reality in v0.7. Existing coverage files will probably appear to skew lower (on average) in v0.7 than they did in v0.6 because of this bug.

Negative Complexities Scores

A new feature of IDA 7 is its ability to parse exception handling information (when present) from the binary. Exception handlers are an implementation specific concept dependent on abstract control flow that is only realized at runtime.

As a result, exception handlers are hard to represent statically in an intuitive manner. IDA 7 presents them as disembodied chunks in the original function’s flowchart (graph view).

Without good heuristics to compute the ephemeral edges to and from these blocks, they disrupted Lighthouse’s cyclomatic complexity approximations and produced comically wrong results for some of these functions.

For the time being, I have chosen to revert to IDA 6.X behavior by simply ignoring these chunks until I have more time to devise a better solution for representing their metrics. This means that these chunks will not be visible to Lighthouse’s painting or coverage capabilities.

In the meantime, HexRays has stated that they are continuing work in this area and will integrate exception handling information into the decompiler(s) at some point in the near future.

Minor Bugfixes

• Fixed a bug that made the pintool unusable (?) for 32bit binaries
• Fixed a regression that caused pseudocode painting to fail on some builds of IDA 6.8
• Fixed a race condition that could crash Lighthouse on rare occasions when saving compositions
• Fixed a bug that prevented compositions from being saved if bc95 was disabled on IDA 7 (non-default)

In the Wild

Since the last release, I have been forwarded a myriad of positive comments about Lighthouse. Among the messages have been links to a few new public works making use of Lighthouse, in the wild.

Microsoft Office RCE, by Embedi

In mid November Embedi highlighted Lighthouse in their blogpost detailing a vulnerability they discovered and reported to Microsoft. Using Lighthouse, they were able to diff coverage sets between two separate runs of Microsoft Office to quickly identify interesting (and buggy) parsing routines.

The Art of Fuzzing, by Rene Freingruber

A week later, @ReneFreingruber published a comprehensive set of resources detailing the tools and techniques that make up an effective (modern) fuzzing workflow.

Lighthouse makes an appearance in the eighth video to diff coverage and identify interesting code (again!)

Conclusion

This was a well rounded release where the features, tweaks, and bugfixes all carried an equal degree of importance towards building a better user experience. The project continues to thrive off the support of the community with a growing number of researchers integrating Lighthouse into their daily workflow.

I welcome external contributions, issues, and feature requests.