Rating Content on Your Website

RFPs for web content management systems often mention content rating as a requirement. I want to explore the topic of content rating, examine its pros and cons and look at options for implementing content rating on your website.

What is content rating? For purposes of this article, content rating refers to the ability of end users to rate content in terms of its usefulness, relevance or overall quality. (This is not to be confused with another understanding of content rating, namely the categorization of content based on its appropriateness for different age-specific audiences.)

Users who read a news story, for example, may be asked to rate the article on a scale of 1 to 5. The rating is typically associated with the user's account (or their browser cookie if they are not logged in). When the user returns to the site, they can see their own ratings and may also see average ratings for all visitors. Rating data may be used to inform popularity indexes (for example, a listing of a site's most popular news stories) or may drive content or product recommendations.

Some content management vendors offer content rating tools either as part of the core product or as add-ons. Vendors may suggest that content rating can help drive decisions about what kind of content to develop or help support content personalization. An underlying assumption in these kinds of claims is that rating information is both accurate and valid.

Much has been written that suggests these rating systems are, in fact, poor sources of information (see here). First, there are questions about the validity of the chosen scale. For example, do all users interpret the scale similarly? Is one user's "Good" another user's "Average"? Even for a single user, scales need to be designed such that individual ratings are consistent. For example, would a user consistently be able to identify an average article as "Average" and a great article as "Great"?

Second, online rating tools tend to use convenience sampling, meaning that the people who actually rate content are those who tend to rate content (not necessarily a good cross-section of your site visitors). Basing content development decisions on a small, non-representative sample of users does not make good business sense.

Many services have emerged to provide alternatives to content rating. One of the leaders of the pack -- Loomia -- focuses on the business outcomes of content rating to determine how to best measure content popularity. Loomia provides a content rating widget that can easily be integrated into any site. Sitecore works with Loomia seamlessly. Loomia integration consists of two steps:

1. Provide Loomia with an RSS feed of site content.


2. Add the desired Loomia widgets to your site.

Both steps can be easily accomplished through Sitecore's RSS module and simple rendering logic. The final product lets users easily add ratings to any content item on your site:

But Loomia doesn't stop with content ratings. As I mentioned earlier, content rating is an unreliable measure of popularity. Instead, Loomia uses rich statistical algorithms that combine ratings, analytics data and other proprietary methods for discerning the value of site content. Loomia takes this data and generates personalized recommendations for users, making your site more sticky (i.e. increasing page views, user sessions and conversion rates).

Here we see two interesting aspects of the discussion brought into clear relief: 1) user-generated content evaluation should focus on business outcomes and 2) evaluation data should be drawn from a representative and valid sample of user behavior. I strongly recommend working with a recommendation service like Loomia when trying to make your content more compelling and discoverable.

CMS and Email Marketing

Email marketing has been, and will continue to be, a powerful way to connect with prospects and existing customers. When site visitors are converted to newsletter subscribers, your chances of motivating them to become customers or repeat purchasers skyrockets.

While the opportunity is great, the risk is also high. First, email marketing is a regulated practice. Due to the unpopularity of spam, Congress has enacted laws to differentiate legitimate email communication from unsolicited and illegal mass emailing. Second, ISPs have gotten into the act by blacklisting emails from origination points that are associated with known spammers. In simple terms, this means that your entire email marketing efforts can be dramatically undermined if AOL, Yahoo! or Comcast decide that you are not complying with the CAN-SPAM act.

For these reasons, I often recommend the use of email marketing services such as Constant Contact or Vertical Response. (Of course, Sitecore also has a newsletter module that is well worth investigating.) Email marketing services help you manage compliance with CAN-SPAM. They also have relationships with ISPs that help reduce the chance that your email campaigns will be blacklisted. If email is so integral to your business that you don't want to outsource the functionality -- for example, if you are sending tens of millions of emails a month -- you may need to go with a homegrown solution. In any case, you should make sure that your campaigns are on the right side of the law and ISP policies so that your marketing efforts will be successful.

How do you integrate your email marketing with your CMS-driven website? The answer will vary depending on which CMS you use. With Sitecore, integration is simple. Both Constant Contact and Vertical Response provide web service APIs that allow the execution of most system functionality remotely. What this means is that you can manage your user lists, subscribe, unsubscribe and even generate reports from within your Sitecore-managed website.

A simple example using Sitecore's demo site is below. Users subscribe using a Sitecore-generated form on the website. This subscription request is then sent to Vertical Response using the Vertical Response API.

This is the simplest possible example -- consider it a starting point for your vision. For example, you can provide a list of newsletters and let users choose which newsletters they want to receive. Because the subscription form is managed in Sitecore, it can be versioned over time so that you can test different messaging strategies and evaluate conversion rates. Your Sitecore form will also be multilingual, allowing users to subscribe to newsletters regardless of their native language.

To populate your newsletter, you need to find the right user interface. Many email marketing vendors require you to use their authoring interfaces to create your email. These interfaces are powerful but, for the most part, do not accept feeds from external datasources. Find out from your email marketing vendor whether they accept data from external feeds, whether they allow you to specify a Sitecore URL or whether they require old-fashioned cut-and-paste.

Constant Contact and Vertical Response are examples of email marketing vendors with open-APIs that connect with Sitecore. If you use a different vendor, you can be assured that Sitecore integration is available if the vendor has a web services interface. Alternatively, vendors may provide pre-built widgets, such as opt-in forms, that can easily be added to your Sitecore solution.

Finally, you might consider working with a Sitecore partner to help you with your email strategy.

Sitecore as a Content Aggregation Engine

Customers often look to Sitecore as an orchestration engine for content aggregation, approval and publishing. Consider the following scenario: A web-based publisher provides information on its site from various sources. News stories are provided from a news bureau; local weather is provided by a web service; selected content is provided from external RSS feeds; some content is authored in-house; finally, product information is provided by an e-commerce engine.

All of this content needs to be presented on the site. Third-party authored content needs to be reviewed and approved prior to publication. The rollout of this content needs to be organized and coordinated either through administrative action or an automated process.

Sitecore dramatically increases developer productivity and business-user control in implementing this kind of a solution. It all starts with careful requirements. A high-level outline for understanding your requirements consists of content discovery, analysis and business rule definition. (An experienced Sitecore partner can help with this process greatly.)

Content discovery refers to identifying all relevant data sources and their attributes such as:

• Data source – Is the content in a database, an RSS feed a web service? Is it managed in-house or at a 3rd party?
• Updates – How frequently is the content updated? How are updates triggered? How are updates communicated?
• Owner – Who is the business owner of the content? Who is the content copyright owner?
• Administration – How is the content currently administered? Who handles the administration?

The content discovery phase is simply about getting clear on what content needs to flow through Sitecore as part of the orchestration process.

Content analysis involves identifying the existing business rules for each content source and determining what efficiencies can be engineered through content orchestration and automation. At this stage of the process, consider the following:

• How frequently are content updates received from external sources?
• Does content need to be approved before appearing on the site?
• How frequently does content need to be updated on the site?
• Are there dependencies between content from different sources?

Developing orchestration rules may be the most complicated part of the process, as it involves detailed inputs from both the technical team and business content owners. Apparently minor details or business requirements may shift the entire direction of the implementation or surface opportunities to achieve unforeseen efficiencies.

Defining business rules involves specifying – for each data source – rules for processing, approval and publishing. Processing rules describe how the content should flow through the CMS – Should the content be imported wholesale into the CMS? If so, on what schedule? How should the content be structured in the CMS? Or should only metadata be registered in the CMS, keeping the original content repository as the true content source? How should content duplication and versioning be handled?

Approval rules describe how content should be handled after being registered with Sitecore, but before being published. Should content be reviewed editorially? If so, who are the approvers? Do they need to be notified of pending content? Have content dependencies been identified? How should these dependencies be handled? How should conflicting dependency rules be resolved?

Publishing rules describe how content should be released to the public-facing website. Should content be released on a scheduled basis? Should it be driven by editorial processes? Will it be driven by dependencies on 3rd-party publishing schedules?

With these requirements in hand, Sitecore is ready to step up to the plate. An experienced Sitecore partner will be able to implement a solution that brings automation and ease-of-use to this otherwise complicated business scenario. Sitecore makes content import, metadata registration, workflow review and publishing straightforward tasks. Without question, these will require custom development based on unique business requirements; but Sitecore’s out-of-the-box features and built-in extensibility will mean implementation and deployment times measured in weeks rather than months.

The result is an orchestration system that puts power, flexibility and configurability in the hands of business content owners, marketers and product managers.

A/B Testing with Sitecore

With A/B testing, marketers optimize their conversion metrics by presenting different marketing messages and testing their results. Wikipedia defines A/B testing as:

A/B testing is a method of advertising testing by which a
baseline control sample is compared to a variety of single-variable test samples. A classic direct mail tactic, this method has been recently adopted within the interactive space to test tactics such as banner ads, emails and landing pages.

Employers of this A/B testing method will distribute multiple samples of a test, including the control, to see which single variable is most effective in increasing a response rate or other desired outcome.

Let's consider the components of A/B testing as they play out in Sitecore. First, we need to define the test. Typically, web tests will consist of at least one of several attributes:

1) Text
2) Graphics
3) Page layout

A text-based test could be testing different marketing messages; a graphical change could include different ad copy, varied product images, etc.; a layout change could place different items on different locations on the page (to determine, for example, whether placement in the center of the page increases sales).

For our purposes, imagine a promotion wherein two different offers are being presented to a customer: 10% off or 20% off. The marketing goal is to determine the price sensitivity of site visitors. Marketers can use different text or graphics to present the offers and then test the effectiveness of each.

There are many approaches to defining tests in Sitecore but the primary driver behind test definition should be usability for marketing authors. You might explore Sitecore's Wizard Framework (requires SDN access) to create a simple wizard for test generation. A simpler approach could be to use content markers to allow authors to create a new test. Consider, for example, the following implementation:

Test variations can be stored as children of a primary content item or can be stored elsewhere in the content tree. Regardless of where the are stored, the implementation should result in a usable experience for test authors.

In addition to determining presentation and content elements to vary with each test, each test will require a duration and a distribution. In other words, how long should the test run for and what percentage of users should get each test (or should the distribution be completely random?). Implementions of distribution rules will vary based on the logic of the rules themselves: Will tests be implemented on separate servers, with distributions handled by the load balancer? Will tests be driven by presentation logic? Will a 3rd-party test engine be integrated with the system?

To evaluate the results of the test you will need to define a conversion event. The conversion will typically involve clicking on a link or a button in a form. You will need to think carefully about what a conversion means for a particular page. Does the conversion occur on this page? Or does it occur on a subsequent page (such as completing a transaction vs. simply adding an item to your shopping cart).

To view conversion metrics, business users can view reports either within the Sitecore Desktop or in a reports module viewable only in WebEdit mode. Conversion metrics should be both clear and actionable. The relevance of conversion metrics will be directly related to the considerations accounted for in the test definition. For help with defining tests, consider working with an experienced Sitecore partner, integrating with a 3rd-party rules engine or reading up on best practices in A/B testing.

Marketing and Publishing with Sitecore CMS

Welcome to my blog on marketing and publising solutions for web content management! This blog focuses on innovative solutions that have been implemented using the Sitecore content management system (CMS).

You may already be familiar with how Sitecore or other CMS's can be used to accomplish basic content editing. Here, we will be keeping an eye out for innovative implementations that help marketers and content publishers broaden their reach and enhance their effectiveness.

If you are not familiar with web content management (WCM), here is a quick definition: Web Content Management systems allow non-technical users to manage content on intranets, extranets and public-facing websites. "Manage content" can mean anything from content authoring and editing, taxonomy management and metadata tagging to controlling security rights, managing workflow processes and publishing. At the end of the day, WCM puts the control of websites in the hands of marketing deparments. Technical resources are often required to maintain the WCM framework, but day-to-day authoring and publishing are entirely controlled by business users.

This blog uses WCM as a starting point for examining innovative implementations that produce unique and effective business results.