Report Update Frequency

Been getting more than the usual number of inquiries/complaints about reports failing to update daily. In fact, the reports are updating as expected, but we haven’t set expectations properly.

There are two main sets of data from which reports are generated. Every day we regenerate each user’s entire “My Followers” report with the data we have collected since the user’s sign-up date (including the previous day’s data, as yet unreported). This data changes every day, so we regenerate the report every day.

The “My Followers’ Networks” report is another story. Crawling a user’s second-level network and calculating the results is a processor intensive operation, but more to the point, it’s expensive in terms of API requests to Twitter for data. Twitter restricts the number of calls we can make, so it’s impossible for us to update the My Followers’ Networks reports daily. We’re working to increase the frequency, but as of now it takes about 3 weeks to cycle through our entire user-base.

Luckily, this data is fairly static. Once you get that first report (and we prioritize reports for new users) the data won’t change much from one crawl to the next.

So if you’re wondering why your report hasn’t been updated in 2 weeks… it has! The frequency-sensitive stuff updates daily. The more static 2nd level stuff may take a while, but it’s OK because there may not be too much new information there when it does update.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: