Simplifying the SAINT Classification Process
Seriously, It’s Not That Complicated
Adobe Omniture’s SAINT tool is pretty basic: it’s a way to organize your campaign initiatives. If you’re sending out your 150th newsletter today, you might want to know if it will be your most successful newsletter ever, so it needs a new tracking code that’s different from the 149 newsletters that went before. But somebody else in the organization may want to know how much traffic came from all of the newsletters put together.
That’s where SAINT comes in. You have one report where each newsletter gets its own value, and another where each value is the same, i.e., “Newsletter”. You upload that table to Adobe, and both business users are going to get what they need.
Okay, It’s not Quite That Simple, Either
For most companies I work for, I’m the face of their analytics system. And when I meet for the first time with online channel managers, such as biz dev, pay-per-click, or email, I usually get a cool reception. Not cool like “awesome sauce”, but cool like they don’t trust me. They have received enough SAINT reports in their inbox to know that they don’t trust them, and they look at me as if I’m going to persuade them why the reports are right and they’re wrong.
The truth is that most of the time the reports are wrong. Not because Omniture blew it, but because the company didn’t effectively carry forward the torch that the implementation team handed them after tracking was set up initially across their site.
There are four major challenges to using SAINT effectively:
- Creating all the right reports
- Developing a scalable pattern
- Uploading values consistently
- Validating the code capture
I’ll discuss strategies each of these challenges in later posts, but right now let me just say that there’s an underlying reason that companies blow it. It’s because there are no rules. Campaign tracking is a blank canvas.
If you’re a botanist and you discover a new species, there is a pretty rigid system in the scientific community for classifying that life form among everything that’s already been discovered. You can’t make up random values for Kingdom, Phylum, Class, etc., or even leave those values blank.
But in the SAINT process, you have to create and enforce the rules yourself. And if you don’t do it with extreme care and diligence, you’ll eventually end up in a meeting where executives are shouting at each other about a channel report, while you wonder silently if they’re really looking at all the right data, and how you’re ever going to regain any credibility at the table if you voice your concern.
The Tipping Point
Being a SAINT table referee is no fun, especially when you have lots of channel managers in the mix, all with their own deadlines and pressures. If you have to generate all of their tracking links for every initiative they sponsor, you’ll start to hate your job as soon as they get a little ambitious.
But the alternative is to train everyone on the rules you developed, and just hope that they follow the rules you set up–which makes that uncomfortable meeting where you wonder if the reports are accurate even more likely to occur, since the whole process didn’t flow through you at all.
Eventually, you’ll either give up and wait for some other hero come in and save the day (for a while, at least, until she gives up too), or you’ll create an tool to automate your rules for tracking code generation and SAINT classifications.
The Quest for Automation
I’ve created lots of automated SAINT systems of varying complexity over the last decade. I started with a shared spreadsheet, and Excel macros that would fill out SAINT templates with the new codes that the user would then save and update manually. Eventually I improved the macros to the point that the spreadsheets were automatically created, saved, and uploaded to an FTP drive that Omniture would retrieve and import.
My most enduring system was an intranet site where channel managers would answer a series of questions about the kind of initiative they were creating, drop in the list of landing pages, and press the submit button. The channel manager never has to worry about tracking code syntax or query string parameters or classifications or Omniture variables.
Sounds good, right? Well, it was good…for a while. But a variety of unforseen, yet inevitable, challenges eventually arose.
The Drawbacks of Automation
Last time I checked, the company was still using the online tracking code generator I built all those years ago. But they were using it grudgingly, and here’s why:
The business questions I asked were no longer relevant. The reporting system as we originally framed it made sense in 2004, but we went through many changes in organization and personnel in the years that followed. Today, half of the original questions don’t even make sense anymore, and the answers people give are the ones their mentor told them to use. That diminishes the value of the reports as well as the tool itself.
The structure is set in stone. Omniture’s canvas is actually customizable; you can add or delete classification reports in the admin interface, and even re-classify your past codes into a new system to make it apply retroactively. But doing so will break the code generator you built, so unless you’re willing to update the automation tool as well, you’re bound to a system that becomes less relevant to your business with each passing month.
Every company requires a custom system. It’s not easy to take the system you built for companyA and apply it to companyB, because companyB has its own campaign organization and reporting framework. Your options are to either push a one-size-fits-all approach across multiple clients, or do a custom rebuild for each one. And if trying to keep a single system relevant to a changing business is hard, imagine maintaining lots of custom solutions!
A Solution Build on SAINT API
The obvious answer is to build something that automatically hooks into Omniture. That way, if something changes in the reporting structure or the way the different codes have been classified, your code generator will recognize the changes and adapt automatically.
It sounds so simple when I put it like that, but I’ve been working on it for over a year, and we’re only now pushing it to beta. What we hope we’ve built is a system that requires no customization to automate the code management you have worked to maintain throughout your organization.
Join our beta group, and let us know if we’ve succeeded!