Yahoo's getting in the game. REST based APIs for Search, Images, News, etc.

Yahoo's differentiators:

• REST based instead of SOAP - this means params are passed in URLs to call the APIs.
• Default quotas of 5000 ops / day for each service.
• Rate-limiting - so you can't call one service too many times in quick succession
• No commercial applications without permission. This is the biggest difference. So is this gonna result in a bunch of toys? Maybe the real players will just get permission to build commercial services...

I think another difference is that Yahoo knows how to support a community launch better than Google. Jeremy Zawodny alone will do more to promote the APIs than the Google AdWords API team has time for. E.g. a quick tour of the sample code shows more depth than the Google AdWords API samples. A nice perl example, even a JavaScript sample. PHP is kind of weak, however...

Paul Bausch, a creator of this fine blogging service (blogger), and author of Amazon Hacks, has a quick Yahoo API sample up on O'Reilly site.

BTW, if you want to get started with the Overture APIs, they are there, you just have to look carefully. Unfortunately, you need to process DTC-XML for Overture. So once people learn SOAP, REST and REST / DTC-XML, we can expect some cool apps!

¶ 2/28/2005

SEMPO releases the results of a survey done in 2004. Click fraud is seen as a minor problem currently.

In your experience, how much of a problem is "Click fraud" with regard to paid placement?

Interestingly, the survey summary claims that advertisers and agencies are currently more concerned about manipulation of search engine results, with repondents averaging a 2.0 on a scale of 1 - 5, agreeing with the statement: "Abuse of search engine optimization practices is a major problem".

My take would be that things change fast in the search world, and it's not surprising that SEO manipulation had more awareness than click-fraud. But I think SEO manipulation is the past trend. There is room for surprise on the click-fraud front since concern about click fraud hasn't reached critical mass in the marketplace. If it does, it is going to have a lot of room to run, since most market participants weren't worrying about click fraud at the time of this survey. Of course, I could be wrong, and click fraud is really nothing to worry about...

¶ 2/27/2005

Here's how to prepare yourself to be able to detect click fraud and potentially get a refund from Google or Overture:

1. Track your conversion rate. Don't just use the Google and Overture tools. Find a unified way to track both conversions from all ad networks, outside of the Google or Overture system.
2. Log everything. Capture referrers, user-agents, cookies, etc.
3. Save everything. Save all your old logs and monthly reports from the ad networks
4. Log a user or session cookie to each web server log line.
5. Log numeric IP address. Don't use DNS lookup to insert the name into your log file.
6. Add tags to your destination URLs - especially on Google! You need to include the keyword and other relevant details. This topic alone deserves a whole post.
7. Graph conversion rate against clicks from Google / Overture.

That's a start - you certainly need to keep good records and all your clickstream data.

¶ 2/26/2005

Seth Godin picks up on some Indian stories about click farms...

Related:The warning at the end of John Heilman's article on Google:

When crisis eventually comes to Google— and it will—the company’s fate will depend on whether they have absorbed a handful of lessons that apply as much to life as they do to business: Adulthood happens. You can’t make all your own rules. And everyone fucks up.

Yes. Within 18 months. Call it the "Come to Jesus" moment. I'm guessing that it'll be related to click fraud.

¶ 2/26/2005

Brian McAndrews in interviewed by the MarketingSherpa folks. He gives four tips, and number three deals with click fraud.

To stay in the game, McAndrews suggests these strategies:

• Look at lifetime value
• First rank isn't always the best
• Continue to monitor for click fraud

If anything, the click fraud topic will heat up this year. Advertisers should monitor campaigns closely: if click volume spikes while conversions stay within a normal range, you may be dealing with fraudulent clicks from affiliates or your competition.

• Search is just part of the mix

McAndrews is the CEO of the company that owns AtlasDMT. So he sees a lot of data. One tip that he probably assumes his audience must be doing is: "Track conversions." That's step one.

¶ 2/21/2005

The fact that Google AdWords rewards relevancy creates a big differentiation for Google AdWords over Overture's system. The simple fact that your ad position is determined by the combination of your bid and your click-thru rate causes you to focus on the right things:

• Keyword selection
• Ad Copy optimization
• Landing page testing and design
• Store usability
• Customer satisfaction

When you try to over-think the buying of PPC ads, you can get distracted by complex bid-management strategies such as day-parting: choosing when to run ads in order to reduce cost.

Although Overture's transparency has some advantages, it's all too easy to get caught up in the bidding process, since the bids are all that control your ad position. Techniques like day parting may make sense for high volume sellers who have very good understanding of their demographics, but for the most part, it's a distraction.

There are better ways to spend your time if you are looking to maximize profit on an ad campaign. Consistently focusing on keywords, ads and landing pages can easily increase conversion rates by factors of 2x or 3x.

¶ 2/19/2005

Brian over at MicroISV speculates:

Now that Google is opening up the AdWords API, my guess is that we’re going to see dozens if not hundreds of apps that will handle scheduling of ads. By scheduling ads to appear at times when people are more likely to download my software, I have a better chance of both increasing sales and quite possibly, decreasing expenses. My other option is to use the API to increase the bids on my AdWords so that they appear first in the list during non-peak times which has the potential to increase sales during times that are traditionally slow.

My view is that it will take a while these apps to show up. Here's why I think the AdWords API adoption is going to be somewhat slow:

• The API is hard to use. SOAP is new to most web developers. Just look at the AdWords API Google Group for evidence of this. There's a lot of stalled tinkerers.
• The quota system needs to change. Right now it's too restrictive and limits the incentive for doing things entirely with the API
• The AdWords community is essentially secretive. Most participants have economic interest in keeping their techniques to themselves.

What could Google do to increase the adoption beyond the AmazEbayShoppingTag.coms? A few things are obvious:

• Change the quotas, so people could use it freely and not have to think about the quota.
• Put out a lot more sample code for people to copy. Publish examples of successful SOAP envelopes, both the requests and the responses.
• Have a contest to promote sharing of code.
• Hire an evangelist who has the time to post on the group and cheer people on...

Note that I don't think the choice of SOAP is a limiting factor. There are some who clamor for a REST version, but I think the AdWords API is more complex than most of the current REST things, and a different interface wouldn't spur adoption any faster. The AdWords API is a great thing because it's an early example of web services for the masses. Google should make it happen faster, though.

¶ 2/19/2005

If this post shows up, I guess you can.

=====
Visit one of my websites:
http://gotads.blogspot.com         -- Successful search engine marketing

__________________________________
Do you Yahoo!?
Meet the all-new My Yahoo! - Try it today!
http://my.yahoo.com

UPDATE: Wow! It worked. I guess I gotta watch that .signature if I want to do that again.

¶ 2/19/2005

If you buy PPC advertising, it's imperative that you track conversions somehow.

If you know your average conversion rate, and you have enough volume, then changes in the conversion rate can be a strong tip-off of possible click fraud.

However, if you are letting Google and Overture track conversions, and they are crediting you automatically for what they identify as fraudulent clicks, then you won't really be able to see what's happening - they are tweaking the data before you see it. Another good reason to track your own clicks and conversions outside of what Google and Overture do for you.

¶ 2/18/2005

I spent an hour hacking the Y!Q javascript to see if I could get it to call Google. Of course in the end I really only changed about 10 lines. The idea was to see if I could get better "contextual" results.

However, I couldn't get the Google results into an IFRAME cleanly. On Firefox, I kept getting a syntax error.

The yahoo javascript grabs the search results from the page with the following line: document.getElementsByTagName("head").item(0).appendChild(oJSResults);

But when you try it against google, it fails with:

Error: syntax error Source File: http://www.google.com/search?q=click%20fraud%2C %20click%20spam%2C%20google%20click%20fraud%2C %20Overture%20click%20fraud&=Search%20Related %20Info&hl=en&num=5 Line: 1 Source Code: <html><head><meta HTTP-EQUIV="content-type" CONTENT="text/html; charset=UTF-8"><title>Google Search: click fraud, click spam, google click fraud, Overture click fraud </title><style><!--

I think the problem may be that <!-- at the end. Google must put that in there for some reason - and it may be somehow messing up the Javascript/DOM ability to grab the right element. So if you know how to fix that, I'd love it if you'd put a note in the comments. Thanks

¶ 2/18/2005

The SoapScope weblog explains how to use their product to see a soap envelope from .NET that's passed over an https service.

I used another method, without using a tracing product, and you can copy the concept from here:

SoapExtension for viewing XML

Basically, you implement a SoapExtension that processes the soap message and you print the output during the AfterSerialize Stage. Fairly painful since you have to add an attribute to every method in the Web reference proxy classes.

However, I do get to see the SOAP output from my service. Unfortunately, I also get an exception:
An unhandled exception of type 'System.Xml.XmlException' occurred in system.xml.dll

¶ 2/16/2005

My previous post tested out Y!Q. Basically the results sucked. Here's why - the "related search" that was done used the following keywords:

email spam google opine contextual search search advertising cause people ppc currency

So basically, Y!Q chooses some words out of the text I provided - which in the case of the last post was all of the text after the first paragraph. The words it choose weren't really very representative.

So here's another try, and this time, I'm providing just the keywords I want:

click fraud, click spam, google click fraud, Overture click fraud

In other words, provide it with the equivalent of a <META> keywords tag. Let's see what happens now...

UPDATE:

google spam fraud click

¶ 2/16/2005

"Click fraud exists, but it's mostly a big paranoia," said Chris Churchill, chief executive of Fathom Online, a San Francisco firm that studies the spending patterns on search engine ads.

Others believe anywhere from 10 percent to 20 percent of the clicks are made under false pretenses.

This article from CNet also cites several sources in the 10-20% range.

But what about last Thursday, Feb 10? Anyone notice 50% more clicks than usual on their Google AdWords campaigns? Random? It wouldn't be too odd if click spam could reach 3X normal levels on a peak day.

In any case, Google's current stance is precarious. Here is a recent quote from the CEO:

"We are always worried about it, but it hasn't been a material issue so far," said Google chief executive Eric Schmidt.

Basically: "Move on. Nothing to see here." Attempting to keep a lid on anything untoward... If the pundits are saying 10-20%, and on a given day it could be 50%+, how long will Eric Schmidt be able to honestly say "it hasn't been a material issue" ? I'm pretty sure he knows who William Lerach is.

PREDICTION: Click fraud will be a HUGE problem for Google this year. Since almost all of their revenue comes from AdWords, the stock could be very vulnerable to any real or perceived threat to that revenue. Google doesn't have THE answer just yet...

¶ 2/15/2005

Here's a transcript of a video of Omri Gazritt at Microsoft explaining the meaning of Web Services and why they are important.

C9: What does Microsoft get out of this? In other words, why should I help Microsoft get richer by adopting, help you get your pay check?
...
OG: Seriously, the more we reach into the enterprise, the more we realized how heterogeneous the environments are. Hence, doing this, for Microsoft helps us be able to sell more of our software to that space. And it's something our customers just demand of us. He says, hey, I have a J2EE environment today on the server. I would love to bridge clients out there except a little detail. All that stuff runs on some other vendor's platform, and your stuff runs on Windows. Can you make it work for me?

SOAP and web services could theoretically make it easy for different apps, databases, web sites, etc. to work together!

All well and good in theory, but I can barely get VB.NET to talk to the AdWords API from Excel VBA, so I kinda wonder who's going to be making their ERP on Sun talk to their J2EE middleware on Linux and have the result show up on the Windows desktop in Firefox...

The upshot, as Simon Fell says, is that "Interop is hard"... True enough, e.g. the PocketSOAP wsdl tool can't grok the AdWords API wsdl without getting "Type Mismatch" errors...

¶ 2/15/2005

I spent a few hours on the phone with some Overture sales and marketing people asking them about their API plans now that the Google API is out. First, Overture does have an API, and it's been in use for well over a year. It's XML based, but not a SOAP interface.

¶ 2/15/2005

Some good advice in this Marketing Sherpa article (free access until Feb 25th) from a training consultant at Overture named Mary O'Brien.

O'Brien says "that advertisers tend to split right down the middle, with 50% getting a better return from Overture and 50% claiming Google offers a better return." Hmmm.

Her tips are good advice, and are similar to suggestions I have for Overture advertisers. Here are the highlights of her tips:

• Tip #1. Repeat the keyword *in the beginning* of the title and the body.
• Tip #2. Include a credibility statement.
• Tip #3. Include price only if you're the low-price leader
• Tip #4. ...or you're the high-price leader.
• Tip #5. Write titles and descriptions that fit into categories.

O'Brien also has some advice on bidding:

Some people say, "I'll choose 15 or 20 keywords and bid till I get to number one," O'Brien says. "That strategy will get you broke.

Read the whole thing - but get there before Feb 25, if you want to see it for free.

¶ 2/15/2005

The first call that you need to really work when using the AdWords API is getAllAdWordsCampaigns. Here's the minimum XML / SOAP that I've found to work. Of course, you need your own AdWordsAPI token and account.

 <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
 <soap:Header>
  <email>EMAIL</email>
  <password>PASS</password>
  <token>TOKEN</token>
  <useragent>AGENT</useragent>
  <clientEmail>CLIENT</clientEmail>
 </soap:Header>

 <soap:Body>
  <getAllAdWordsCampaigns>
    <dummy>0</dummy>
  </getAllAdWordsCampaigns>
 </soap:Body>
</soap:Envelope>

¶ 2/14/2005

Having devoted / diverted my own attention for the last two weeks to the Google AdWords API, and away from my regular business, I think this article is a must read for any marketing manager thinking about using Google's AdWords API or even Overture's API.

The bottom-line is nailed down right here:

Before looking at a build-versus-buy decision, consider the following:
Your business rules set. Having access to the back end of Google or Overture is useless if you don't know what your goals and objectives are nor how those objectives can be coded into a flexible system that uses feedback to make adjustments and changes in the engines.

¶ 2/10/2005

In other words, the person in charge needs to manage a mature set of keywords, understands conversion, cares both about PPC, organic, email, etc. They have to help their product managers, content people, and sales people understand search, and they have to keep them happy (if someone calls from the Barcelona office and wants to know why no ad shows up under some Spanish language term, they don't spend a lot of time explaining why, they just add it to the list - because it's cheap and easier than arguing about it)

In sum: prioritization, reducing risk, keeping things predictable and integrating with other marketing efforts while keeping the rest of the company happy is their main task. Search will mature and be managed like everything else, and software vendors will have to address all the pieces.

PS. What's the hardest problem for these F500 guys? Finding good consultants who can integrate into the WHOLE marketing strategy and focus on the business value while monetizing search. If they find one, they are either hired or hard to keep interested.

¶ 2/10/2005

When does it make sense to use Overture?

• You can get more clicks - a 10% click rate is easier to attain
• You can reach different people (like moms, and non-geeks) easier
• You can write longer ads
• Less competition
• It's easier to understand how to bid

¶ 2/09/2005

Giving the search engines visibility into the effectiveness of keyword buys could potentially allow them to consciously inflate their rates – Google and Overture will recommend a company's most successful words to their competitor, setting off small bidding wars.

Bottom-line: Don't give your data to the ad network. Someday it may be used against you. Use 3rd party or in-house tracking tools instead.

¶ 2/09/2005

In the olden days, I used to work with a great programmer named Phil Karlton. We worked on a real-time OS that ran on an SGI Indy and tried to make it become a set-top box for interactive TV...

However, in the early days, nothing was stable at all. Not the tools, the OS, the server side, the database - NOTHING. Each morning was a two hour fun-fest of fixes to your personal sandbox before what you had yesterday would come close to working.

Some days, you'd call Phil over to understand why your build was broken, or the code didn't work. He'd look at your versions, and look at the problem, and then usually say: "It works for me."

Which was basically a curmudgeonly way of saying: get on the right train, use the right versions, and do what I'm doing if you want anything to work...

So although Phil passed away in a tragic accident, I think I'd know what he'd say about working with the Google AdWordsAPI. If I were smart instead of stubborn, I'd have the same development environment as the Google team. As near as I can tell that would be:

• Linux
• Java
• Eclipse IDE
• Apache Axis with SAX 2.0

¶ 2/08/2005

An otherwise prolific programmer who runs KillerSite.com bemoans the difficulty of SOAP in VBA (that's Visual Basic for Applications, ie. Excel Macro language)

I'm having quite a time working with MSSoapLib... I searched all over Google and Google Groups but have found nothing helpful...

The best place I've found for help are the xml and Excel VBA forums over at p2p.wrox.com. However, much of the Microsoft world is moving away from VBA towards Visual Basic .NET. The big advantage of VBA is you don't need a several thousand dollar investment in Visual Studio .NET to write a program that sends 400 bytes of xml over http. The disadvantage is that you can't get anything to work, and it's hard to debug... at least for me.

¶ 2/08/2005

"For several months now I’ve been troubled by a nagging and unpleasant thought that there is a potentially large vulnerability in Google’s Adwords business model. ... A billionaire has arranged to give $100m to the first person that clicks on a special link that looks like a Google text ad."

I.e. the billionaire attempts to get a mass number of Internet users clicking on every ad, in order to find the "golden ticket"... AdWords advertisers would find they were paying for huge numbers of clicks that had no intention of converting to buyers. A panic could ensue, and everyone would stop investing in AdWords campaigns.

This is interesting to think about, but I think it misses a key point. Google has created a market - much like the stock market. Since the AdWords market does have intrinsic value to a lot of players, it would be hard to "corner".

If someone offered $100M dollars to incent mass number of users into click-fraud, it would shock the market for a short time, but the market would quickly "discount" the attempt. In other words, the advertisers would recall they can still derive value from placing ads, and actual buyers would still find their sites. In response, they could simply lower their bids to reflect the influx of non-converting clicks.

Google could also adjust quickly to such a threat, by lowering minimum bids requirements, for example.

So in the end, everyone's conversion rate might go down by a factor of 10 or 20, but the actual number of conversions wouldn't be affected too much. So you'd still spend the same on ads to get the same result.

I do think it's an interesting question - and it could definitely affect the stock market's view of the company. Remember when Jim Clark announced his confidence in Netscape by saying he would invest $30M of his cash back into the faltering stock? That move bolstered NSCP quickly enough so that Clark's overall stake increased in value by over $100M...

Via GoogleBlogoscoped. Philipp notes the other interesting part of this thought experiment - Google could eventually move toward a pay-per-conversion model, rather than a pay-per-click.

¶ 2/06/2005

Best of all, she starts finding products for her customers instead of finding customers for her products.

¶ 2/05/2005

Here's a test program I wrote. You can indeed call the Google AdWordsAPI from JavaScript...

function postData(Url, Data, Action) {
  var oHttpReq = new ActiveXObject("Msxml2.XMLHTTP.4.0");
  oHttpReq.open("POST", Url, false);
  oHttpReq.setRequestHeader("Man", "POST" + Url + "HTTP/1.1");
  oHttpReq.setRequestHeader("MessageType", "CALL");
  oHttpReq.setRequestHeader("Content-Type", "text/xml");
  oHttpReq.setRequestHeader("SOAPACtion", Action);
  oHttpReq.send(Data);
  return oHttpReq;
}

function main() {
  var sUrl = "https://adwords.google.com/api/adwords/v2";
  var sData = ' <soap:Envelope  xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns="https://adwords.google.com/api/adwords/v2">
<soap:Header>
<email>EMAIL HERE</email>
<password>YOUR PASSWORD</password>
<useragent>XXX</useragent>
<token>YOUR TOKEN</token>
<clientEmail></clientEmail>
</soap:Header>
<soap:Body>
<getOperationCount>
<startdate>2005-01-02T23:59:59</startdate><enddate>2005-02-02T23:59:59</enddate>
</getOperationCount></soap:Body></soap:Envelope>';

  var oResponse = postData(sUrl, sData, "getOperationCount");
  if (oResponse.status == 200) {
    WScript.echo(oResponse.responseText);    
  } else {
    WScript.echo("Could not retrieve data:\n" + oResponse.statusText + " (" + oResponse.status + ")");     
  }
  oHttpReq = null;
}

main();

¶ 2/04/2005

• PocketSOAP - soap interfaces for Windows that actually work!
• Soapclient.com: A list of soap web sites
• Soap specs at w3.org
• SOAP::Lite - Perl's primary soap toolkit homepage
• XMethods.net Directory of Webservices - good for testing

¶ 2/04/2005