Democratic Research

So – you need to automate important marketing processes in your business, and have begun researching which vendors provide the software your business will require for optimal functionality and strategy. 

The good news? You’re not the only one who has travelled down that research path, and Research Reports can be an excellent source of information supporting your marketing automation decision. 

The bad news? These report shouldn’t be taken at face-value, it’s important to understand the behind-the-scenes of each to determine just how big a grain of salt you are reading.

There is a multitude of research report types so let’s categorize them at the highest level. On one end of the spectrum, you have the Analyst POV Reports with industry analyst expertise and in-depth research. On the other end, we have Crowd-Sourced Reports in which rankings are driven by the quality and quantity of user reviews. 

Analyst POV Reports

Pros: The research and evaluation processes here cannot be understated – the “Tier One” industry analysts who do this work are experts in their field and seriously know their stuff. They sit through strategy and product presentations/demos and some even get feedback from referenced customers. The vendors invest days of time and resources to provide the right information to the analyst. Of course, many also sign up as clients and engage with the analyst on an ongoing basis to optimize the relationship. 

Spoiler Alert: In my time as Research Director at Forrester, I had an analyst in my team who only advised and consulted about how to execute the process of Analyst Relations (it’s part of B2B Marketing after all) – including how to get yourself placed in an optimal position in a quadrant or wave analysis.  

Cons: The Analyst POV Report is offered to the research firm’s clients, which are usually large enterprises (that fact also influences the list of vendors included, of course). These are smaller communities than is often assumed. Usually, the readership of each report behind their paywall is perhaps in the hundreds – one vendor client told me that the latest two reports where his product was featured had 480 and just 58 views on the research website.

That can be a little depressing to the analyst – all that work and so little attention. Of course, the brand power, and resulting product-marketing ego, of being in a “Magic Quadrant” or “Forrester Wave” means that some vendors buy reprint-licenses and offer a download of the report through their  website. And they book the analyst to make speeches/webinars about the research – a little show business that compensates for the initial disappointment perhaps. 

Some of the vendors are just happy to be in the report, regardless of whether they are positioned well or not. Here is an example of someone who bought a license –even though Gartner had rated eight vendors higher than them in Execution capabilities. 

Crowd-Sourced Reports

Pros: It’s probably also helpful to seek out feedback from other users; peers who share the good, bad, and everything in between about a product. There are several such feedback websites now up and running for all types of software applications, including marketing. 

Cons: Have you ever looked up your favorite restaurant on Yelp, noticed a few one-star reviews, and wondered how they could come to such contrasting conclusions? A single review (good or bad) shouldn’t dictate your software-buying decision, just like with any other product. Remember: User opinions have varying levels of actual marketing automation understanding–that is to say, just because someone writes a review does not make them an expert in the field.

Additionally, report rankings are driven by the quality and quantity of user reviews. If a company has a few hundred reviews with a high rating average, and another has a few thousand reviews with above-average ratings, it is likely the latter will position better in the report due to the sheer number of reviews. This is a huge advantage for larger vendors that have been on the market for a long time, and it’s likely they have review incentive programs to boost their ranking.

Well, What About Something In-Between ?

We’ve created a different methodology at my business partner Research in Action. We survey 1,500 practitioners about THEIR view of a vendor’s product, service, value-for-money, and ability to innovate. The vendors who score highly enough in the survey qualify to get into the Vendor Selection Matrix™ report (usually 15 to 20 vendors). 

Then, that curated market feedback is flavored with a touch of industry analyst expertise to provide a more well-rounded recipe for successful vendor selection. In fact, much more than the quadrant or wave reports, these reports are embellished with several pages of trends insights that inform both buyers and vendors alike about what is most important when investing in the upcoming project. 

Democratic Research is In-Between and Open 

When Research in Action publishes its (my) reports, they are made available to several communities, for free:

• Survey respondents. The 1,500 marketing software decision-makers who answered the survey questions are provided with the full report as feedback
• Survey panel. Research in Action maintains an active survey panel on a global basis with contact details and topics of interest: a current total of 90,000 Marketing Automation decision-makers. These panel members are informed of the report and can download it if desired
• Website visitors. Any viewers of the Research-in-Action website sees a “public version” without the exact scores and matrix placements of each vendor (to save their embarrassment) but with all insights and the most important facts on each vendor.  
• Vendor reprints. Research-in-Action does also license reprints, where a vendor can distribute a copy of the report, including their detailed profile, to interested parties. 

As an analyst, I am proud that so many people now get to see my work. And, when I am booked to do speeches and webinars, I realize that they are booking me personally, not the brand power. 

The Vendor Selection Matrix™ reports do fill that gap between an industry analyst report focused on large enterprise needs, and the “trip-advisor” type of review websites. They also reach and assist a broader community of software buyers. Lastly, the community reading the reports is probably a whole order of magnitude higher than the audience able to access the “Tier One” research reports.    

Here is our latest Market Impact statistics chart.

Always keeping you informed! Peter