Twitter: Please solve your search index problem

The Problem:

Twitter’s real-time search index is not tracking everyone’s updates, and their people search is not showing everyone either. This has been reported at Twitter support, but they have not posted any news or fixed the problem for over a month.

How to check

To see if you are impacted by this, go to Twitter Search and search for: “from:yourusername”
without quotes, and with your Twitter username instead of “yourusername”

• See the results for my username: Twitter Search “from:FreelanceLance”
• Compare that with my Twitter profile, which shows that there are in fact tweets from me.
• Here are two people with ~8,000-10,000 followers who are in the same boat:
  • Peter Pek: profile |  search results
  • CJ Robinson: profile |  search results
• Actually they are worse off because they have the double whammy: they also don’t show up in People Search, as you can see: CJ Robinson |  Peter Pek
  There are results in both cases, but they do not include those two users. (I’m lucky enough to still be in people search for whatever reason.)
• Update: I just brought a restaurateur friend onto Twitter (@mainstreeteats) and he is immediately not available in People Search. See here.

Lots of other people are having the same or very similar problems, as you can see in this support thread at Twitter (which has been open since June 4) about the tweets missing from the public timeline/search, and this one (open since May 29) about the people missing from People Search.

Why it’s a problem

It’s pretty easy on the surface to see why this isn’t good, but it might not be quite as obvious why it’s a big enough deal to bother doing something about. There are a lot of ramifications for the Twitterverse that spring from this problem.

People who are not included in Twitter’s search index:

• are not indexed at hashtag-tracking sites like hashtag.org and wthashtag.org
• are not included on other sites that rely on the search database:
  • Specialized sites like (for example) 311 Tweets or industry-based directories
  • General Twitter tracking sites like Twitalyzer (which won’t add you to its index unless it can find you in Twitter’s)
• are not able to really play along with fun hashtag games, including #followfriday (plus #coolbandsmadeuncool, #1stdraftmovielines and so ons)
• are not able to really participate in important and/or topical hashtag threads (like #iranelection
• are not able to include themselves via the self-indexing function of hashtags (i.e., #tlot) or using event-based ones (like #nctweetup)
• in many cases, can not be found by name via Twitter’s people search
• can not be found in searches based on keywords
• in many cases, can not be seen when they @message someone, unless that someone is already following them
  • result: people can be giving props or thanks or saying something negative, and the @target of those words is not made aware of it
  • another result: people can be asking questions or trying to communicate with “big people” (celebs, etc.) or companies, and not be heard, and not be aware that they aren’t being heard
• are not able to enter contests or promotions that are hashtag-based (and may not even know it – see the Moonfruit example).

The bottom line is that until the search index is fixed, a large population of twitterers being arbitrarily excluded from a lot of the functions that make twitter so cool and useful. Not only are they at a disadvantage in trying to grow their network and spread their message, they are unable to do everything they could be doing for and with the Twitterverse–which does a disservice to everyone on Twitter, and to the natural organic growth that should be occurring in the community there.

The worst part about this is that these “hidden” folks have a much harder time even getting this issue out there, since they can’t hold a hashtag-based awareness campaign, and in many cases can’t be heard by the “influentials” on Twitter in order to get the word out. And Twitter seems to be pretty silent on this issue.

So it’s up to everyone to realize that this problem affects all of us on Twitter, and to demand that they make addressing this problem a top priority.

NOTE: Yes, I know Twitter is free, but let’s not kid ourselves: the people who created and run Twitter are going to make a great deal of money, and it is the “twittizenry” that has brought about the growth and momentum to make that the case. We are the value in Twitter; and I’m pretty sure the people who run it know that.

We just need to let them know what’s important to us. In this case, ending the second-class status that has been applied for whatever reason to some unknown population of Twitter users. Or to put it less dramatically:

Dear @Twitter: Please solve your search index problem. (Plz RT)

Ideally, they will respond quickly and let us know what’s up, and we won’t have to apply much pressure. They may even have a solution in the works, and all that we need to do is get some loud enough voices to ask them what’s going on.

That’s where you come in.

What to do:

• Tweet about this problem and spread the word so that everyone knows about it. (Include @Twitter in your tweets, but don’t start the tweet with it.)
• Start a #fixTWsearchplease(or something like that) hashtag campaign and try and get it into trending topics
• Ask twitter celebrities and influencers (people with big/loyal followings) to spread the word and to directly pursue Twitter themselves.
• Get in touch with groups and sites that depend on Twitter search or hashtags. Everyone who uses hashtags to organize tweeters is impacted by this.
• Check to see if you are included, and if you aren’t, get suitably perturbed about it. Tell people of your perturbery. Perturberate profusely. Also pontificate and/or pronounce, if needed.
• If you find that you or anyone else isn’t being indexed, post a message at Twitter’s support forum : here if your tweets are missing from the main search, and here if you are missing from people search. (Starting a new ticket seems to be a waste of time, as they just send generic auto-replies, according to multiple accounts.)
• If you notice someone else isn’t included, let them know. The more people who know they are being impacted by this the better. Ask them to help spread the word.
• If you have extra time, you can check to try and find high-reach twitterers who are not being indexed. They have the best chance to bypass the imposed silence and get the word out directly. And they will have motivation to do so. (Also see above about places and tweeps who rely on hashtags/search.)
• Publicly tweet to @Twitter about this. Examples:
  • Dear @Twitter: Please Fix Your Search! http://bit.ly/lInXj (Plz RT)
  • Your search isn’t working right, @Twitter! Can you tell us when it will be fixed?
  • I’m not in @Twitter’s search! What’s up with that, Twitter? http://bit.ly/lInXj
  • My friend @myfriend’s not in @Twitter’s search! What’s going on, @Twitter?
• When tweeting the word about this, be sure not to start your tweets with “@”-someone, or it will be hidden from most people’s timeline. You can just put a dot in front of it if you want; this will make sure people see that this is being talked about.

If you have any other ideas or know of a better way to get the search fix moving along, and/or get Twitter to be more forthcoming about it, please post them in the comments below. (But if you know of someone missing from search, make sure you post about it at Twitter’s support ticket (here and/or here first.)