Some Background to my Vendor Research

If you are reading this, then I assume that you’ve looked at a couple of my Vendor Selection Matrix^tm reports and are thinking … they look like magic quadrants or waves but they seem to be different… Well, they certainly are – in more ways than one !

I do this work with my business partner Research In Action and mine are written for marketing software buyers who need to automate one or more important marketing processes and are researching which vendors COULD provide the software their business will require for optimal functionality and strategy. 

They’re likely to be calling the project something close to the process(es) being automated and improved, but there is no guarantee that the vendors will be using that terminology when describing their products.  

I design my projects around the process name I think Marketers would use and survey businesses on their experience. Often, that collects a landscape of vendors using different technology labels but that is the reality.

All in all, there is a multitude of vendor report types out there. On one end of the spectrum, you have the Analyst Reports with industry analyst expertise and in-depth research. On the other end, we have Crowd-Sourced Reports in which rankings are driven by the quality and quantity of user reviews. 

Analyst Reports

Pros: The “Tier One” industry analysts doing this work are experts in their field and seriously know their stuff. They sit through strategy and product presentations/demos and some even get feedback from referenced customers. Vendors must invest days of time and resources to provide the right information to the analyst. Of course, many also sign up as clients and engage with the analyst on an ongoing basis to optimize the relationship. 

Spoiler Alert: In my time as Research Director at Forrester, I had an analyst in my team who consulted specifically on how to execute the process of Analyst Relations (it’s part of B2B Marketing after all) – including how to get yourself placed in an optimal position in a quadrant or wave analysis.  

Cons: The Analyst Report is written for the research firm’s clients, usually large enterprises – which influences the list of vendors include, of course. These are smaller audiences than is often assumed. Usually, the readership of each report behind their paywall is perhaps in the hundreds – one vendor client told me that the latest two reports where his product was featured had 480 and just 58 views on the research website.

That can be a little depressing not only for the vendor but also for the analyst – all that work and so little attention! Of course, the brand power, and resulting product-marketing ego, of being in a “Magic Quadrant” or “Forrester Wave” means that some vendors buy reprint-licenses and offer a download of the report through their website. And they book the analyst to make speeches/webinars about the research – a little show business that compensates for the initial disappointment perhaps. 

Crowd-Sourced Reports

Pros: It’s always helpful to seek out feedback from other users; peers who share the good, bad, and the ugly about a product. There are several such feedback websites now up and running for all types of software applications, including marketing. 

Cons: Have you ever looked up your favourite restaurant on Yelp, noticed a few one-star reviews, and wondered how they could come to such contrasting conclusions? A single review (good or bad) shouldn’t dictate your software-buying decision, just like with any other product. Remember: User opinions have varying levels of actual marketing automation understanding – just because someone writes a review does not make them an expert in the field.

Additionally, report rankings are driven by the quality and quantity of user reviews. If a company has a few hundred reviews with a high rating average, and another has a few thousand reviews with above-average ratings, it is likely the latter will position better in the report due to the sheer number of reviews. This is a huge advantage for larger vendors that have been on the market for a long time, and it’s likely they have review incentive programs to boost their ranking.

Research In Action Reports Have Both Perspectives

The methodology at Research In Action is that we first survey 1,500 practitioners about THEIR view of automating the process(es) in questions. And then we ask them to name one or two vendors they associate with the project and give us feedback on the vendor’s product, service, value-for-money, and ability to innovate. The vendors who score highly enough in the survey get into the Vendor Selection Matrix^tm report in the first place (usually 15 to 20 vendors). 

Then, that curated market feedback is seasoned with a touch of industry analyst’s (that’s me) expertise to provide a more well-rounded recipe for successful vendor selection. In fact, much more than the quadrant or wave reports, these reports are embellished with several pages of trends insights that inform both buyers and vendors alike about what is most important when investing in the upcoming project. 

Research In Action Reports are Widely Read  

When Research In Action publishes its reports, they are made available to several communities:

• Survey respondents. The 1,500 marketing software decision-makers who answered the survey questions are provided with the full report as feedback
• Survey panel. Research In Action maintains an active survey panel on a global basis with contact details and topics of interest: a current total of 125,000 IT Automation decision-makers and 90,000 Marketing Automation decision-makers. These panel members are informed of the report and can download it if desired
• Website visitors. Any viewers of the Research In Action website sees a “public version” without the exact scores and matrix placements of each vendor (to save their embarrassment) but with all insights and the most important facts on each vendor.  
• Vendor reprints. Research In Action does also license reprints, where a vendor can distribute a copy of the report, with their detailed profile, to interested parties. 

On average, each report gets tens of thousands of clicks on our website. Personally, I am quite proud that so many people now get to see my work. And, when I am booked to do speeches and webinars, I know they are booking me personally, not the brand power. 

Our work really does fill that gap between an industry analyst report focused on large enterprise needs, and the “trip-advisor” type of review websites. They also reach and assist a broader community of software buyers. Lastly, the community reading the reports is probably a whole order of magnitude higher than the audience able to access the “Tier One” research reports.  Here is our latest Market Impact statistics chart.

Always keeping you informed! Peter