I want to give credit to Alex Payne, Matt Sanford, and Doug Williams of the Twitter API team.
They are a true pleasure to work with, they are extremely responsive, and they are very helpful.
Without their help, TweetLater would not be where it is today.
^DP
Up to now we have used several URL shortening services that provided you with no means to track the clicks on the links in your scheduled tweets and DMs.
We have now added bit.ly as a URL shortening option. All you need to do is add your bit.ly login and API key to your TweetLater profile. From that point forward we will shorten all your links using your bit.ly account. We won’t use any of the other URL shortening services.
This means you can track the clicks on all your links in your bit.ly account.
To activate bit.ly on your TweetLater profile, do the following:
It’s as easy as that. From that point forward all your links will be shortened using bit.ly.
This will also soon include also includes links on tweets created by your Blog Feeds, if you are a TweetLater Professional user. We have not yet now added it to the blog feeds processor.
The same two bit.ly fields are also available on the registration page, for folks who are registering new TweetLater accounts.
If you don’t have a bit.ly account, it’s super easy to register a free account at bit.ly.
If you don’t know where to get your bit.ly API key, login to your bit.ly account and click the “account” link in the top right corner of the page. Your API key is displayed on that page. Your “login” is the username that you use to login to bit.ly.
Friday, April 17, 7:44 PM EST.
If you’re seeing a lot of entries in TweetCockpit that say the Twitter API returned zero tweets, or if you have trouble with the Vetting page, or your tweets are not being published on time or not at all, or if you’re getting a message on the Twitter account add or edit page about invalid Twitter credentials, please note the following.
Both the Twitter API and the Twitter Search API seem to be very unresponsive at the moment.
We assume it is a fallout of the new Mikeyy worm attack that struck today.
It is not a problem in the TweetLater system. It’s just that we’re not getting data back from the two APIs.
We’re even seeing it on the batch jobs that run on two different servers. They have all slowed down to an absolute crawl, and are getting tons of fail whale errors back from Twitter.
Update April 17 9:14 PM EST — Just heard back from Alex Payne at Twitter. The Twitter system is completely overloaded at the moment. It was featured on Oprah earlier today, and is right now on Larry King. That’s why the APIs are not responding.
Update April 18 6:42 AM EST — The APIs are still extremely unresponsive. Some of the API calls are going through, but the majority are still getting fail whales.
Update April 18 12:22 PM EST — As noted in the Comments, I am waiting to hear back from Twitter. Earlier this morning I have asked them to check whether their spam team has perhaps put a block on my IP address. There is a remote possibility that someone could have scheduled tweets that the spam team considers as spam. They are not supposed to block my server IP address in that case, because it does not make sense to inconvenience 40,000+ people because of some stupid thing one person did. However, I have asked them to check whether that is the case. Personally I do not think this is the case, because some of the API calls are going through. There is not a complete blackout of calls from my server IP address. With a spam block the blackout is usually 100%, i.e., no API calls get through. Twitter is on PST, so I expect to hear back from them later this afternoon.
Update April 18 7:20 PM EST — Have not heard anything back from Twitter. It is extremely frustrating. However, let us remember that it is weekend.
Update April 18 8:27 PM EST — It appears as if the Twitter folks are still combating the new Mikeyy worm. Alex Payne and Matt Sanford of the API team are usually very responsive to my requests for assistance. I’m sure I will hear back from them the moment they are able to stop working though code and can check their emails. I’m convinced that in the Twitter offices combating the worm takes priority over everything else.
Update April 19 6:56 AM EST — The problem still persists, and Twitter has not responded yet.
Update April 19 10:36 AM EST — My head is now really very sore from repeatedly banging it against the wall. We should probably all resign ourselves that this issue will only be resolved on Monday during PST business hours. My sincere apologies to more than 44,000+ of my users who have been inconvenienced by this issue. If there were anything more I could do to have it fixed I would have done it days ago. ^DP
Update April 19 7:00 PM EST — Nothing new to report yet, except that the bruise on my head is now turning purple and black around the edges. For a third-party application like TweetLater to have an API issue, which essentially cripples the service for several days, with no support, is really not an acceptable situation. Even though it’s not done maliciously, it is an issue that Twitter will have to address. ^DP
Update April 19 9:05 PM EST — I saw Alex Payne, head of the Twitter API team, online and tweeted him. He has just replied with, “Got ’em [my emails]. We should have it sorted for you tomorrow [Monday].”
Update April 19 10:11 PM EST — You may notice a slight improvement in TweetCockpit, adding and editing Twitter accounts, and Vetting. The API issue still exists, but I thought of a crude work-around, which I’ve now implemented. It may still be spotty, meaning it may work only intermittently. At least it’s better than nothing.
Update April 20 3:36 PM EST — The issue has now been resolved. TweetLater was unfortunately caught in the cross-fire between Twitter and spammers. With the latest round of spammers and worms that have hit Twitter, they have severely tightened their API behavior analysis algorithms. Due to the very high volume of Twitter API calls that TweetLater makes around the clock (we make several calls per second), one of the spam traps in the tightened algorithms was unfortunately triggered. As they say, **it happens. I have added additional code on my side to try and prevent this from again happening in the future. I will extend trial periods before end of day today. Follower processing will, over the next 8 hours, resume where it left off on Friday. ^DP
Update April 20 5:16 PM EST — A further update regarding follower processing. The normal 8-hour cycle of follower processing has resumed. Your account will be processed within the next 8 hours, and all the followers that followed you since Friday will be picked up. However, I’m seeing that the API is slow, which could be just because it is very busy or because the unblock is still filtering through all of Twitter’s networking equipment. The API usually speeds up late at night when the load is not so heavy. I’m expecting that things will return to normal within the next 24 hours. By then the TweetLater system should have caught up with the backlog that has built up. ^DP
The problems you’re having with TweetLater will automatically go away the moment that the Twitter API and Twitter Search API again start responding.
The trial periods of folks using the TweetLater Professional trial will be extended as soon as the issue is resolved. These extensions will be have been done between 5:20 PM EST and 5:50 PM EST.
Update April 20 9:57 PM EST — This issue is now officially resolved and closed. Follower processing has a HUGE backlog and will take 24 to 48 hours to catch up. At the time of writing there were 1.7 million unprocessed API calls on the processing queue, and the queue is still growing every second.
I want to express my sincere thanks to the Twitter API team, and to Matt Sanford in particular, for the efficient and professional help in getting this issue resolved.
Update April 21 10:38 AM EST — I have now employed a second server, which will process new API calls from the top of the processing queue. Hence, you may find that new followers of today are processed before folks who followed you during the outage. The outage has created a backlog of nearly 4 million API calls and it will take the system a few days to work through that backlog. ^DP
Update April 21 5:33 PM EST — Working through the backlog is going smoothly. There are now only 2.3 million API calls left. Piece of cake. ^DP
Update April 22 10:30 AM EST — It is official. The system has now finished working through the backlog of nearly 4 million API calls. We are now fully caught up, and just normal processing will take place from now on. At various points during the catch-up process, we were making in excess of 30 calls per second to the Twitter API, and the API handled it without blinking an eye.
The TweetLater website was unavailable for roughly 20.5 hours, from around 4:00 PM EST on April 16, 2009 to 12:30 PM EST on April 17, 2009.
All scheduled tweets that were supposed to have been published during the downtime have been placed into an error condition. They will not be automatically published. You need to review each one and decide if you want to reschedule it, or if you want to delete it.
Follower processing has resumed where it stopped when the server went down. In other words, followers who have followed you during the downtime will be processed. It will take roughly 8 hours from 12:30 PM EST to cycle through all the Twitter accounts.
We have already sent credits to the two people whose profile banners were affected by the downtime.
All payment information sent by PayPal to our system during the downtime have been manually applied to the user accounts.
If any TweetLater Professional customer requires a partial refund for the downtime, we will be more than happy to oblige. On payments of $19.97 the credit is $0.57, and on payments of $29.97 the credit is $0.85. Please submit a support ticket and let us know if you want a credit.
We added two additional days to all TweetLater Professional trials that were active when the server went down.
The TweetLater.com server is currently not responding.
We’re working on getting it back online as soon as possible.
If you happen to get to the login form, and it says your credentials are invalid, it’s because the database is down.
We will update this message when service has been restored.
Update 9:24 PM EST — The faulty drive is being replaced on the server. Unfortunately it was the primary drive, which holds the operating system. The OS needs to be reloaded, which they tell us will take between 6 and 8 hours.
Update 10:51 PM EST — The server rebuild is being started now. The database has been backed up, so there should not be any data loss. Follower processing and scheduled tweets are currently in mid-air suspension. When we reactivate the batch jobs once the server is rebuilt, follower processing will resume. All unpublished scheduled tweets and DMs that should have been published during the downtime will be published at that point. You may experience a sudden publishing of several tweets at the same time in your timeline if you had several tweets that should have been published during the downtime.
Update 11:04 PM EST –The hosting folks say that the server rebuild will take 6 to 8 hours. We’re going to assume a horizontal position for a few hours. When we’re awake and fresh, we’ll check that everything has been correctly configured on the server before we open TweetLater.com for logins. Will update here again in the morning.
Update 5:08 AM EST — The server has been rebuilt, and the backups are currently being restored.
Update 6:55 AM EST — Backups have been restored. There has been no data loss, apart from a few Keyword Alert archives that were corrupted beyond repair. It shouldn’t be an issue because those archives are only kept for 24 hours anyway. There are a few more configuration tasks left to do. It will probably be another hour, maybe two.
Update 7:02 AM EST — After doing an analysis of the scheduled tweets that should have been published during the downtime, we felt it would be prudent to put those tweets into an error condition so that they do not publish, and that you can decide whether you want to delete them or reschedule them. There are many Twitter accounts that will have a small avalanche of tweets published on them all at once if we just let the system publish the tweets. We’ll make a separate blog post regarding these tweets as soon as the system goes live.
Update 8:33 AM EST — It’s still going to be a while. Apache is still recompiling. Once that’s done, there are still a few configuration tasks to perform.
Update 9:14 AM EST — Apache recompile has finished. Now doing the last configuration tasks. It won’t be much longer.
Update 9:55 AM EST — Nearly there. Some of the batch jobs are already running, so you will see scheduled tweets being published. The site is still down because MEMCACHE is not yet running, and without it the site simply unceremonially crashes.
Update 11:01 AM EST — The hosting provider technicians forgot to compile several essential libraries into PHP. Just waiting for them to do that, then the site should come back online.
Update 12:01 PM EST — Just learned an important lesson. HostGator works on the tickets that have the oldest customer reply time first. When you keep submitting replies, such as “Please work on this, it is really important,” you bump your own ticket just further down the queue because the reply is new. So we learn a new thing every day.
Update 12:30 PM EST — And…….
WE ARE BACK!!!!
The big site header graphic at the top of each TweetLater page has been bugging us for a while, because it was taking up so much screen real-estate.
The site header has now been reduced in height, which allows you to see more above the fold on every page of the site.
The font used in the header has also been changed. The new font, we believe, looks a little more “professional” than the old one.
Last night a real rookie mistake with regex (regular expressions) caused some of our follower snapshots to become corrupted. Really badly corrupted.
The system is currently rebuilding all follower snapshots from scratch (April 10th, 2009 9:00 AM EST). This process will run until 12:00 PM EST.
The nett effect is that folks that followed you before 12:00 PM EST will not be processed. In other words, they will not be auto-followed and they will not receive your DM.
After 12:00 PM follower processing will resume as usual.
The rookie has been sent to stand in the village square so that all the children can ridicule him.
For the time being we have disabled the Friends/Followers counts that were displayed for each Twitter account on the List Accounts page.
The Twitter API was returning 0 Friends for certain accounts, which made the display rather useless, and the API has been extremely slow the past few days.
The slowness in the API resulted in the List Accounts page being very slow to load.
Once the API is back to its usual speedy self, we will re-enable the display of those counts.
Since Twitter has changed their display of @replies to also include what they now call @mentions inside the tweet text, we have decided to do away with the separate @replies digests that we used to send out.
However, don’t despair. They are not gone, they are now just integrated with the keyword alert emails that we send out.
If you had the @replies digest option enabled on your Twitter account entry, you will notice one of the following things happening:
1) If you had existing keyword alerts, your @replies will now be included in those emails. You don’t need to do anything.
2) If you did not have existing keyword alerts, the system has now created one for you with your @username as the keyword. We’ve set the default delivery frequency to once every 12 hours. You can change the frequency if you want to. Simply login and click the Keywords menu tab.
This change took effect on April 9th, 2009. You should receive your first email of @replies before the end of the day.
As you probably know, when we launched TweetLater Professional we also built in a limit that you could add a particular Twitter account only to one TweetLater account.
That worked well for some folks, but quite a few had the need to allow others to also manage their Twitter account in TweetLater.
Take as an example Joan, who wanted to schedule tweets on her business Twitter account, but she also wanted Kelly, her assistant, to schedule tweets and manage the Blog Feed that posts the website’s RSS feed entries to the Twitter account.
Joan had to give her TweetLater password to Kelly, which was far from ideal because Joan also had her personal Twitter account in her TweetLater profile.
We have now added the ability to delegate the management of your Twitter account in TweetLater to as many people as you want. The only requirements are that you must have a TweetLater Professional account and so must the people you want to delegate to.
Now Joan can keep her TweetLater password a secret, and still give Kelly the full management rights to the business Twitter account. Kelly also does not need to know the password of the Twitter account.
As soon as Joan delegates the account to Kelly, she will see the account magically appear in her TweetLater list of accounts, and she can just start adding tweets. She can even manage the account in her TweetCockpit.
This new feature is also great for companies that want several employees to manage a Twitter account.
Take the case of a media company that has a VP of TV Stations, several regional managers, and the station heads of the different TV stations.
With this delegation, the PR Department can create a Twitter account for each TV station, and grant access to it in TweetLater to the VP, the regional managers, and the station managers.
The VP will have access to all the TV stations’ Twitter accounts, the regional managers will have access to the stations that fall under them, and each station manager will have access only to his station’s Twitter account.
They all can see what tweets the others are scheduling on the account, and they can add or modify as they see fit.
However, none of them ever need to know the passwords of the Twitter accounts. They only need to know the password of their TweetLater account.
Revoking access is just as easy. One simple click of a button and the account disappears from the other person’s portfolio of accounts.
That makes employee on-boarding and off-boarding a breeze because you simply grant or revoke access. You don’t need to change your Twitter password because it wasn’t known by the employee who has resigned.
It is truly a multi-person integrated management feature.
This new feature is included in the low monthly subscription price of TweetLater Professional.
If you already have a Professional account, then you already have access to it. There is a link to it on the List Accounts page.
SocialOomph Blog 
You must be logged in to post a comment.