Partial TweetLater Outage Due To Twitter Denial Of Service Attack

Early on Thursday morning Twitter suffered a denial-of-service attack.

As part of their defense against the attack, they blocked a large number IP addresses, including, we presume IP addresses that were making high-volume calls to their API.

TweetLater’s IP addresses were also blocked in the process, since we normally make more than 40 API calls per second in the normal course of our business.

TweetLater was not part of the denial-of-service attack. We were simply caught up as innocent bystanders in Twitter’s defense against the actual attackers.

As part of the recovery from the attack, Twitter is unblocking the white-listed IP addresses of third-party applications such as TweetLater.

This unblocking process is a meticulous one and there are many IP addresses to unblock (not only TweetLater’s). According to Twitter engineers, the process can last well into Saturday.

We do not know exactly when the actual unblocking of TweetLater’s IP addresses will occur, but we do know they will be unblocked.

We appreciate your patience and understanding.

Normal service will resume as soon as the IP addresses have been unblocked.

Free trials of TweetLater Professional will be extended by a few days once normal service resumes.

Update: This thread on the Twitter Developer Talk Group shows that many other third-party applications were also blocked.

Update Thursday, 8:25 PM EST: We have now restored normal operations. We have also extended all active free trials of TweetLater Professional by one day.

Update Thursday, 8:38 PM EST: We spoke too soon. Sorry. Services have been paused again. Twitter is not out of the woods yet.

Update Friday, 12:24 PM EST: Some features of the online website have been restored. All automation processes are still in paused mode, because the Twitter API is still refusing to process any high-volume API requests.

Update Friday, 10:38 PM EST: There has been no change yet. The Twitter API is still refusing to process any type of volume API requests. Twitter has communicated that the denial-of-service attacks have been ongoing and have intensified on Friday, and that they believe those attacks are geopolitical in motivation. Twitter is still defending against the attacks, and has asked everyone to hang in there with them.

Update Saturday, 00:19 AM EST: The latest update from Twitter is that there is no, we repeat, no ETA on when these issues will be resolved.

Update Saturday, 8:12 AM EST: The status has not yet changed. The TweetLater web site is operational, but all Twitter automation is still paused. Blog feeds and Ping.fm automation are processing as usual. Please login to TweetLater and read the service announcement above the blue menu tabs for more details on what is paused and what is running normally.

Update Saturday, 5:44 PM EST: There has been complete and deathly silence from Twitter today. No communication about the status to application developers. We have cautiously turned on the process that publishes scheduled tweets to Twitter. There may be periods where it may be blocked, during which it will revert back to putting tweets that are older than two hours into an error condition. But, so far so good…

Update Saturday, 6:00 PM EST: Nope… As soon as you send even a small volume to the API their edge defenses simply block your IP address. It is high time that Twitter identifies who are their friends, and allow them to operate normally while defending against the bad guys. They already white-list our IP addresses, so they know exactly which IP addresses are approved ones that should be allowed to work normally.

Update Sunday, 10:55 AM EST: The status has not yet changed. All developers of third-party applications are still waiting for an update by Twitter regarding correction of the severe limitations currently imposed by the Twitter API. The last time Twitter has provided any meaningful update was on Friday.

Update Sunday, 1:54 PM EST: The status is still the same. Twitter has published an update saying that the attacks are still ongoing, and that they are still working on restoring normal access to the Twitter API. There still is no timeframe for the resolution of this issue.

Update Sunday, 4:29 PM EST: The Twitter API has now been opened for volume transactions. It appears that the issue has been resolved, and we have resumed normal operations. Existing free trials of TweetLater Professional have been extended by three additional days. Thank you for the patience and support that you have shown.

Advertisements

39 thoughts on “Partial TweetLater Outage Due To Twitter Denial Of Service Attack

  1. pfmonaco

    Boo! Was counting on RSS_Feed update as well as PING.FM updates to use TweetLater to updated Facebook / FriendFeed / Brightkite. Perhaps with future events you'd consider only shutting down twitter feed, possibly toggle the pause flag on them and allow RSS and Ping.FM to continue? With PodcampBoston this weekend was especially counting on some help w/automation even if Twitter remained down.

  2. Jasper Silvis

    I really appreciate your detailed note on your main page. It took a while since I have many twitter accounts, but was able to reschedule all my tweets for Monday afternoon per your suggestion. Thank you for suggesting postponing automated tweets, instead of just giving us an update on the situation with Twitter. You are a class act looking out for the best interests of your clients.

  3. david loughnan

    Appreciate your advice. feel real sorry that the hackers have got in a caused you all this greif as your service is great. hang in there guys, there are numerous folks wordwide who support you…you'll come out the end better for it and better prepared for the future. Although right now you are probably tearing your hair out and wondering why me ? Rwst easy, your supporters understand and will remain so for the long run. Good luck folks… From David in Australia.

  4. Dewald Pretorius

    I will definitely do that in the future. It will just require recoding of some of my processes. Unfortunately, right now, when I switch off things, I switch off everything. I will make it more granular for when issues happen again in the future. Thank you for bringing this to my attention.

  5. guidoaugustin

    Well done Dewald, keep it up! It is not about complaining about Twitter, it is about standing together to keep what we all love!Yours Guido

  6. shackbase

    Like everyone else pointed out – Thank You for keeping us all updated; not only re: the TweetLater services but also on the background at what's happening with Twitter.Suggestion – if this were to happen again in the future, how about notifying all TL users with an email similar to this post?Anyways – keep up the great work – it's appreciated!

  7. dspaventa

    Thanks for keeping us informed. Please know that I greatly appreciate the fact that you are always transparent with these type issues, whether it be the shutting down of accounts by Twitter last month or this attack yesterday. Thanks for taking the time to keep us informed.

  8. drd4U

    Look- one of the easiest ways to avoid a DNS again is to expound the site – duplicate site centers – uh, let me see…disenfranchise the netui to many physically diverse farms? Does that make sense…tell you where the code is…remember the massively dispersed array that came out of Berkley on SETI? You could use a similar, “hey let's help ..” kind of approach? It has it's problems, but one that it avoids is DNS…they'd have to DNS the whole web…I know…”everybody loves a smart eleck…” πŸ˜‰ luv you guys at twitter later…

  9. drd4U

    Yes you are absolutely right, it's refreshing to hear and experiment in truth site management!

  10. drd4U

    Appreciate your honesty, BUT – it's about 1:30 AM CST – and I'm on Twitter and I know I'm watching some others do auto feeds? Somebody has a system that is pushing content? Not fussin' just sayin'drd

  11. lhaisma

    Thanks for keeping us posted! I think it is good to give Twitter the time to fight this off. I wish other auto-posters would stop their services too so Twitter can have the time to fight this off and take appropriate measures. I only post to Twitter if it's really necessary at the moment. Come on, we can do without Tweeting over the weekend? Being in IT myself I know there are some guys/gals with sweat on their back working their … off to get this fixed. Respect!Kudos Tweetlater and Dewald!Hope things are resolved soon and we can use your fantastic service again.Have a nice (offline?) weekend everbody!

  12. BlazingMinds

    I dare say the Twitter is doing their utmost to get everything back up and running 100%, I can now use Tweetdeck for most of my tweeting, can't follow via the twitter website, can on tweetdeck, also having problems accessing sites that use Oauth, but things do take time!

  13. Inetcoach

    I also appreciate your up to date news and explanations of the challenges twitter has;I was getting a little frustrated as there are hundreds of new followers not getting followed back,I can't follow them manually at the moment or thank them for following.It's very reassuring to know you're on the ball and time will sort our predicaments….Look forward to that though :-)Thanks again for your efforts.

  14. Bryan Longworth

    I'm having to use a desktop program to send Tweets. Tweeting from TweetLater is much easier. Thank you for your updates and for doing your part to resume services.

  15. DenisCampbell

    Thanks for letting us know what's going on. Twitter (and Facebook) is (are) arrogant and completely unresponsive to customer queries. Being judge, jury and executioner works only for awhile. Hope they stay on top because when you piss off customers and something better comes along… They both have lots to learn in the customer care/PR areas. Keep up the good work. Glad I did not upgrade to Professional this week as planned. Will when everything back to normal.Best,

  16. drd4U

    Haven't seen a Sunday update yet, but am waiting to get after ONE – some BONO insights into the need for Ethiopia, have loads to do, do see some folks who evidently are autobotting? But anyway…am waiting and patiently hoping this site will come back around, any news would be appreciated…

  17. calebdecj34

    Yeah, I sure wish I had a plan b, this is hurting me big time. I did look for other services like tweetlater, but tweetlater is the complete package. You would think that after 3 days we would have figured something out.

  18. stevepohlit

    I appreciate all the updates. What is a bit confusing is that I see some high volume Tweeters still in action. For example looking at Chris Voss, his tweets surely seem to be largely automated and they are flowing. I thought he was using Tweet Later. Certainly could be that he has a staff flowing tweets manually but it doesn't look that way to me.Also just using Tweet Deck it feels like there is quite a bit of volume going thru Twitter. All of this answers nothing and just adds to the perplexity.I do know that communication with Twitter and other social media site user support people is generally a joke. That just keeps this whole industry wide open for a quality competitor.

  19. Dewald Pretorius

    You need to look at the “from” source of the tweets in the Twitter web interface. Chris is making all his updates manually from Twitter desktop or phone clients.

  20. Jerry

    Your situation is a difficult one. As you fully understand it is also difficult for us. When the service is returned do you have any plan to provide some discount for loss of service? I am not meaning this in any antagonistic way. I just think that its good business sense to offer some sort of relief. If Twitter was completely broken, then I would not ask. But currently we have shifted to posting manually and its working fine. I know that your App can't connect, but now I am incurring cost to handle this manually, and I am still paying you for a service your unable to provide (regardless of the reason). The price you charge is very small so this is not really an question of price, rather of principle. And again this is more a theoretical question based on an open discussion policy you promote. If you said NO I would not have a problem with that answer at all. Please advise.Thank you

  21. glider_rider65

    Yes, unexpected benefits like realizing I hadn't picked the vegetable garden in several days… Keep up the great work sir…. Appreciate all you do.

  22. Dewald Pretorius

    We gladly provide credits when we messed up. However, we will not be able to run a business if we had to provide credits every time that Twitter has an outage. Twitter is definitely not going to reimburse us for losses during an outage, which is not under our control or even within our sphere of influence.

  23. webgoddess

    I wouldn't be too sure about that. YouTube has been going downhill fast and nobody has been able to come out with something better to replace them.

  24. jerry

    Fair enough! Thank you for responding. Your open discussion policy is a breath a fresh air! I asked a tough question, and you published it publicly with a rational response.We are still seeing intermittent issues with TweetLater which we fully attribute to Twitter, but at least its working again. Given the fair cost of your pricing, your open communications, constant updates, and refreshing conduct of your business, I am very much a fan of your service. Thank you!

Comments are closed.