Venture philanthropists and social investors have long struggled to measure the true impact of the organizations they fund.

But a new Web-based evaluation tool, the Portfolio Data Management System (PDMS), promises to allow donors and investors to track their investments’ performance and let them, along with the social enterprises they fund, compare their data to those of organizations doing similar work. Salesforce.com, which granted 5,000 nonprofits free licenses to access its customer relationship management software platform, is working with Acumen Fund to rebuild the software and aims to make it available on that platform in January.

Brian Trelstad, Acumen Fund’s chief investment officer, began developing the system after he joined the social investment fund four years ago. “We were tracking different metrics month by month on various spreadsheets, so it was hard to know what was happening in the portfolio,” Trelstad remembers. “We were giving stars and everything was a four out of five—like in Lake Wobegon, all the children were above average. And any comparison of our investments across the sector was based on thin slivers of public information we got from our peers, little of which was quantifiable or verifiable.”

With financial support from the W.K. Kellogg Foundation and Cisco Systems, Trelstad and his team searched for existing software that could help them compare their investments’ performance over time, against their projections, and compared to a relevant peer group. But nothing had the capacity to track the operational, social, and financial data they were collecting from their enterprises.

“On the venture capital side, most software tools focus on the financial metrics, and the grantmaking tools don’t let you look at that many quantitative metrics.” They then decided to ask Google, a donor to Acumen Fund, for help. (Google engineers can devote 20 percent of their work hours to any projects they like.) After four Google engineers spent 18 months working with Marc Manara, an Acumen portfolio associate and recent computer science graduate, Acumen had a software tool it hoped would manage and track its portfolio’s performance.

To use the software, a portfolio manager logs in and creates a new investment record. Together with the management team of the business, he or she then identifies 10 to 15 financial, operational, social, and environmental metrics that are important to running the organization and that demonstrate social or environmental impact. Portfolio managers are also encouraged to select a few common metrics (say, revenue growth or jobs created) that enable across-portfolio comparisons. Every quarter the managers collect those performance metrics and enter them into the system. They add qualitative information—monthly reports and an annual capacity assessment survey—to balance out the quantitative data, and the portfolio team then uses the whole package to assess the overall effectiveness of the enterprise, how it is performing, and what Acumen Fund can do to support the organization better.

The tool’s capacity to measure performance against peers will come in about 18 months, after Acumen finishes working with PricewaterhouseCoopers to decide on the final data classification system and select the right software to aggregate the data across portfolios. But how well that piece performs will depend on the input and use of many social investors and foundations. Trelstad and the Acumen Fund team are rallying their peers to participate.

“Until we build a data system that uses comparable metrics and a wide range of investors are willing to contribute their data into a shared pool, we will never be able to compare the performance of our portfolio companies in a relevant peer group,” Trelstad says. “If 25 social investors, each with a portfolio of 40 to 50 companies working in the same markets collaborate, we might all generate real insights into what is working and what is not from those 1,000 or so investments.”

So far, Google.org and Acumen are using the tool; the Skoll and W.K. Kellogg foundations, as well as 20 other organizations including Root Capital and E+Co, are beta testing it; and some 100 foundations and other social investors are “kicking the tires,” as Trelstad puts it. “We don’t want people using the system until the data are secure and stable and we’ve found a critical mass that sees real value in the tool.”

The verdict so far? “This has really helped our team have better, more informed discussions with our donors and investees,” says Trelstad. Acumen fund managers now use the data in the PDMS to make follow-on investment decisions, and in several cases, the system has signaled performance issues. Portfolio associate Vikram Raman, for instance, working out of Acumen’s India office, noted a significant rise in the operational expenses over the prior quarter at VisionSpring, a nonprofit that offers affordable reading glasses to the poor. VisionSpring then used the data to explore potential causes and solutions with Acumen’s India management team.

As to what other users think, “the biggest negative feedback was that there weren’t enough reports, and that people could define things differently—there’s a huge cultural part of this,” Trelstad reports. “But mostly people told us this is amazing.”

Read more stories by Jennifer Roberts.