DummySpit

Jim Nail of the US media evaluation provider TNS is arguing against the use of the three evaluation categories - Output , Out-take and Outcome - as being too generalised and non-specific. Here’s the link to his blog, http://blog.cymfony.com/2007/10/outputs-and-out.html. These terms were developed by Walter Lindenmann in the early 1990s as a neat categorisation and have been promoted through many books and industry/academic papers.

Output is the production of the public relations effort (messages sent). This is the Presentation stage and may include measurement of press releases and other communication methods

Out-takeor Out-growth is the understanding and retention of messages by publics. It looks at the attention, comprehension and acceptance.

Outcome is the effect of the PR effort on attitudes, opinions and behaviours. This measures the actions of the target groups.

There is also a more recent Out-measure, called Outflow that considers the build of value by improvements to reputation and the creation of organisational/stakeholder relations.

This group of Out-measures is convenient and, as far as I can see, widely accepted. What’s your view?

After three months of discussion, the Study of the Priorities for Public Relations Research (PR Priorities Study - final report) has been completed. The initial piloting was undertaken on DummySpit in April and led to the setting of 26 public relations topics. These were sent to a Delphi study panel (of experts) in five continents covering top academics, leading practitioners and the CEOs of PR industry bodies. After three rounds of intensive email debate, the Top Ten PR research topics are:

1) Public relations’ role in contributing to strategic decision-making, strategy development and realisation, and organisational functioning

2) The value that public relations creates for organisations through building social capital, managing key relationships and realising organisational advantage

3) The measurement and evaluation of public relations, both offline and online

4) Public relations as a fundamental management function

5) Professional skills in public relations; analysis of the industry’s need for education

6) Research into standards of performance among PR professionals; the licensing of practitioners

7) Management of corporate reputation; measurement of reputation

Ethics in public relations

9) Integration of public relations with other communication functions; the scope of public relations practice; discipline boundaries

10) Management of relationships

Just outside the top ranked priorities are:

11) Client/employer understanding of public relations

12) The impact of technology on public relations practice and theory.

This report is the first completed international study on public relations research priorities (using a Delphi panel) since the mid-1990s and gives valuable insight into the ‘front and centre’ public relations research areas around the world.

The results will allow academics and practitioners to work closely together to improve understanding of public relations and its most effective and ethical use. It is a benchmark that all research plans and funding can be judged by for relevance and importance.

David Phillips has just posted a podcast on his site about ROI and the measurement of online PR. It follows on from earlier discussion on DummySpit about “pullability” as a measurement of the associative referencing of online communication. From this discussion, the concept of “out-pull” as a measure was proposed. It has received a very positive reception from practitioners and academics.

Is ROI a relevant measure? Listen to David’s views. As he says, analysis shows that the measurement of online PR and communications is not an easy task and not given to financially-based measures.

On a helpful new blog at the Institute for Public Relations, Sean Williams of Goodyear in the US describes low cost methods of judging the effectiveness of the tyre maker’s internal communications, especially on its intranet. 

They have used internal discussion groups to quickly evaluate the corporate intranet and a daily poll question of key topics. It has built dialogue with staff, with around 150 joining the intranet discussion groups and 800 a day taking part in the poll. Some 450 a day check the poll results. 

The outcomes of the Intranet review have been very constructive – “we did get opinions that differed significantly from those we brought to the exercise. Our intranet is better for the experience”, says Sean. 

Although he says the polling methods aren’t scientific, they are a good example of how “little and often” can build up a period-on-period picture of communications. By using internal resources and a mindset set that allows modification, it can be as valuable as formal research methods. 

In Paul Noble and my book, Evaluating Public Relations, we recognised that many communicators want to research but are stumped by cost or lack of knowledge. That’s why we have shown how media evaluation can be set up using a simple Excel-based spreadsheet or even a paper-based clerical method. The information obtained immediately assists media relations activity. 

Doing evaluation is not hard and it gives immediate benefits – Congratulations to Sean for sharing Goodyear’s approach.

Writing negative comments about your organisation on a personal blog is seen as ‘ethical’ by almost 50% of PR people. That’s one of the results of an international survey by Prof Don Wright and Michelle Hinson reported in the proceedings of the International Public Relations Research Conference, which have just been posted online.

The survey of PR practitioners found that 45% the respondents are aware that employees of their company or a client’s organization have communicated on weblogs, and that the outcomes have been mostly positive. As to whether it is ethical to monitor employee postings and to discipline staff, there was a strongly positive view. Some 79% said that monitoring staff was “ethical” and 59% backed discipline for staff who write negative statements, unless the staff were representing a labor organisation or trade union.

Despite these strongly held views, only 3% actually undertook research or measured “information their employees are blogging”, although 46% expect to do so in the future. Wright and Hinson also point out there is little research into the impact of blogs and other social media on the theory and best practice of communication. Obviously, there is a debate to be had on the ethics and legal issues that surround employee blogging.

The debate over ROI and accountability has taken an interesting turn in the Advertising column in this week’s Media Guardian. The advertising grandee Simon Marquis reviewed a new analysis of advertising effectiveness, Marketing in the Era of Accountability, written by Les Binet and Peter Field, and published by the Institute of Practitioners in Advertising.
The focus is wholly on marketing and advertising but has interesting messages for those involved in PR and related marcoms activities. Most notably, it is about how ROI and accountability have not necessarily improved advertising effectiveness.
Marquis says: “The obsession with accountability has, the IPA says, actually prompted marketers to pick on single things that can be easily measured. In truth, effectiveness is not predicted by any one metric but by a whole range of them. As we have long suspected, success in advertising ain’t that simple.
And that is the case with PR and marcoms, too. The search for the ’silver bullet’ of a simple, single metric or a financially-based construct called ROI doesn’t give the answer to effectiveness in below-the-line communications either.
Marquis continues: “A proper focus on profit-generation makes the report authors a bit sniffy about the fashion for ROI (return on investment) in marketing. I’m not convinced the two are incompatible. Indeed, Antony Young, president of Optimedia in the USA, concludes in his new book, Profitable Marketing Communications, by saying: ‘Marketing ROI is not just an alternative term for effectiveness, impact or results. Nor is it a magic equation or formula. It is an attitude about creating profit.’”
To paraphrase Simon Marquis, “success in PR ain’t that simple”. This column recall the CIPR’s 2005 paper that said evaluation and measurement are complex matters in all organisations that aren’t solved by simple, simplistic metrics. Too true!

http://media.guardian.co.uk/mediaguardian/story/0,,2105117,00.html

David Phillips kindly sent me a posting on the role of ROI in Marketing from Brian Carroll’s B2B Lead Generation Blog -  http://blog.startwithalead.com/weblog/2007/05/the_difference_.html - which discusses the difference between ROI and marketing accountability. As this is an issue in public relations, I have posted this comment:

Part of the problem with ROI is that a financial concept is applied to a non-financial activity. Sure, marketing and sales activity should result in financial results but the misuse of specific business language in an effort to get understanding from the Board is only piling pressure on marketers and communicators.

One of the definitions of ROI is the ratio of how much profit or cost saving is realised from an activity against its actual cost, expressed as a percentage. In reality few marketing or communication programmes can be expressed in that way because of the problems in putting a credible financial value to the results achieved. In 2004, the UK’s Institute of Public Relations said, “this (use of ROI in PR campaign) is not only confusing but misleading” when the term PR ROI is used loosely. Unless the objectives of the activity are solely to achieve a sales or financial outcome, ROI is meaningless.

For marketers, the application of ROI limits their role to sales support and ignores the brand and reputational issues. In PR, I’ve long argued that the use of business language is a fundamental sign of insecurity and a lack of confidence. It seems that marketing has the same affliction.

To read more on my views on the role of ROI in public relations, see this article on the PRism online academic journal,  http://praxis.massey.ac.nz/fileadmin/Praxis/Files/Journal_Files/Issue3/Watson.pdf

The latest announcement from Cision, the owner of the Delahaye Index, says that Microsoft has the “best reputation in the media” based on (according to the Cision press release of May 7) “a score of how many positive and negative reputation driving attributes are found within each story”.

It goes on to breathlessly to say that Cisco Systems ranks second because of “high profile coverage derived from Cisco’s negotiations with Apple over the name of the iPhone” and acquisitions. Following on is General Motors which has roared up to the top three having been “ranked in last place” a year ago.

Dummyspit isn’t quibbling over who is first, second, third or tenth. That’s irrelevant although league tables are a much-used tactic for creating media coverage. Our question is over the misuse of the term, ‘reputation’ and the claim by Cision North America’s CEO Steve Newman that “corporate reputation is uniquely measurable through the media, as news affects and reflects public sentiment.”

That’s wrong. A generation of study has suggested that media may have agenda setting properties but it doesn’t necessarily shift public sentiment. And it won’t always reflect public opinion. That’s wishful thinking by PR practitioners and service providers and is a classic example of the substitution game – where output is confused with outcome.

Reputation, as this blog has said before, is given to organisation by those with whom it is engaged. The media is just one of those stakeholders. It can reflect a positive or negative sympathy towards an organisation and there’s no doubt that media analysis can judge tone and favourability. This factor (sympathy, tone or favourability) is an affective component but leaves out the cognitive element. It is thus worth noting the comments of Prof David Dozier, one of the gurus of public relations research, who commented that reputation may be based on direct experiences as well as on processed messages. (Dozier 1993: 230)

Following on from my posting of March 23 (Online PR Evaluation – do we need new models?), there has been a wide range of comments. Some agreed with my point that Walter Lindenmann’s “out-take” descriptor was the most likely zone of measurement that could be achieved in measuring online public relations activity. Others said that was all wrong as the online community didn’t want to be “messaged” and that organisations had to lay in wait to be discovered by searches (i.e., it is a passive presence). Katie Paine illustrates this in her posting on May 9, “you need to be there when people are trying to find you …”

Going back to Lindenmann, it’s worth reviewing what he wrote about out-takes, even if it was conceived in the pre-online era (that is, about 15 years ago). To quote from Watson & Noble (2005: 83), out-take measures “judge whether or not the target audience actually received the messages and so evaluates retention, comprehension and awareness.” In the online era, the problem is that it is difficult to know whether the audience is reached because it is amorphous and ever-changing. The point I was making was that we may only be able to measure the manner in which the audience uses the “messages”, information or resources in blog, news media or corporate form to pass it on to others. This is equivalent to the out-take judgements.

In an email to me, David Phillips argues that “pullability” is key factor to measure. His draft definition is, “it is the result of activity by the online community combined with technologies to aid a process of pulling information that is “out there” for use by an actor.” He adds that evaluating and assessing the reach of an organisation’s online presence isn’t possible because there’s no way of identifying the total readership of online content. “At best one can monitor a handful of web sites and access research data about online traffic.”

He goes on to argue that “there is evidence that content evokes behaviours online”. And it is by measuring when someone refers to a press release, web page, video, keyword, concept, tag, blog associated with an organisation in an online technology such as a blog¸ podcast, video, wiki or other channel, that evidence can be found that content is being “pulled” by the online community.

Comparing the two notions of “out-take” and “pullability”, I would argue that they are similar but not the same. Out-take is concerned with message recognition and the interim responses before there is any decision to act (or reject). Pullability refers to recognition of information which is found and then passed to others, without action necessarily resulting. Both terms recognise a mid-way stage in handling of messages and/or information.

So my proposal is that “Out-pull” may be the appropriate term to describe the process of handling information and messages. It fits into the Lindenmann model which is a neat and compressed way to explain the stages of PR measurement and evaluation.
 

On April 20, I asked DummySpit readers for comments on a list of research topics for public relations. Some 24 were listed and you could nominate up to ten of them for a prioritisation of the topics that practitioners and academics consider should be investigated.

Here is the first round of results that have come from Australia, Canada, India, Ireland Switzerland, United Kingdom and the United States. Respondents were practitioners, academics and industry people. They worked in consultancies, government, universities industry, non-profits and suppliers.

1 The impact of technology on public relations practice and theory
2 The measurement and evaluation of public relations, both offline and online
3=  Integration of public relations with other communication functions; the scope of public relations practice; discipline boundaries
3= Management of corporate reputation; measurement of reputation
5= Client understanding of public relations strategy and tactics
5= Ethics in public relations
7= Research into standards of performance among PR professionals; the licensing of practitioners
7= Professional skills in public relations; Analysis of the industry’s need for education; Theories of practice
7= The place of “word-of-mouth” and buzz marketing in public relations practice
10= Strategic planning of public relations programmes
10= Quality of public relations services
10= Crisis management and communication; issues management

Just outside this top group, which numbers twelve and not ten, were: Management of relationships, stakeholder approaches, negotiation and conflict resolution; The definition of public relations; International issues in public relations; Cross-cultural public relations; The expectations of users of public relations; The client: consultancy/adviser interface; and the role of PR in community/social responsibility programmes.

It is sad but not surprising that the old chestnut of measurement and evaluation ranked second, despite the welter of widely available information and research on the subject. In Delphi studies in 1994 and 1997, it ranked as top priority for research. Not much has changed in a decade.

In addition to the topics that were originally proposed, several others were suggested, including:
- The place of communication and PR professionals in the boardroom
- Recruitment of better quality entry into the profession
- Public relations as a fundamental management function
- The role of public relations in contributions to strategic decision-making, strategy development and realisation, and organisational functioning
- The part public relations plays in the creation of value for organisations, through building social capital, managing key relationships and realising organisational or competitive advantage
- Social media and its role in public relations
- Identification of the areas of cross-over with, for example, HR, social marketing and especially the management function

Or is a more global view needed, such as “looking ahead ten years, PR will need to have a number of over-riding elements to survive. We have to be able to manage uncertainty (notably in communications channels), change, risk and opportunity. We will need a portfolio of relationship capabilities to hone research, skills and technologies.”

Now for Round 2 - Would you replace any of the top priorities with one or more of these additional topics? Please join in the discussion with your comments on the first round of ranking of the topics and suggest the research questions that should be asked on these topics. For example, 65% of respondents agreed that “the impact of technology on public relations practice and theory” was one of their priorities but what are the questions that we should be asking? That question can also be applied to all those priorities.

Over to you and your feedback.