Request for suggestions

Jun 15 2011

It’s been a while since I had the time to really do something constructive with the code, the project has been put into a maintenance phase, not moving in any direction, as I’ve been caught up in work — quitting employment, beginning a new.

I’ll put this request for suggestions out; what would you like to seen added and/or changed on MediaCreeper? All suggestions will be taken under consideration, I’ll even blog about each suggestion and evaluate it, with argumentation for/against inclusion into the code base.

Issues that is already on the roadmap..

  • Optimize database-connections (in transitional work for moving off MySQL to a non-sql database in the future, probably MongoDb)
  • Make web site more dynamic, add ajax to update listings
  • Clean up code base to prepare for public release, as open source
  • Sync API-calls with WordPress plugin CreeperPing
  • Adding full support for IPv6 — currently spotty support

.. there is possibly more issues that I’ve forgotten about, adding them as they bubble up.

No responses yet

Better IPv6

Apr 25 2011

Just updated a chunk of code that handled the parsing of IPv6 (6to4) .. where IPv4 are tunneled through IPv6 … and IPv4 addresses are embedded within IPv6 addresses. The chunk of code has been tested against approximately 16000 IPv6 addresses and test patterns.

Possibly we have missed a few hits where a small number of addresses were parsed wrong and dismissed as negative hits. We will keep an eye on the statistics for this and tweak it further if necessary.

CreeperFramework (which is the platform MediaCreeper is built upon) does not yet support IPv6 fully, work is in progress for this and will be supported in the near future. CreeperFramework will be released as open source when code base is mature for a public release.

No responses yet

Old logs

Apr 17 2011

To correct and fill in missing data, I wrote a small shell-script to parse through old logs, some 9Gb of old logs. There is some more work required to massage the parsed data to something useful, so it’ll be online as soon as I’ve had a few more hours to spend on it.

When done, it should be possible to render diagrams of traffic on MediaCreeper from November 16th 2008 to today’s date, on total hits, positive hits and unique IP’s.

The last couple of weeks has been quite busy, so I haven’t had that much time to work on CreeperFramework and MediaCreeper. I have a few new ideas to try, but I’ll put them on hold for the moment.

The end of this month will be the last days on my current job, a new job from the May 2nd which is based in my city, less travel, better salary, more sleep and more time for MediaCreeper.

No responses yet

CreeperSiteMap

Mar 30 2011

Somewhere along all updates I forgot to update the CreeperSiteMapEngine, perhaps one of the most important pieces for search enging robots to index, it generates an XML with links, priority, update frequence for almost all pages on the site, again it was a silly error .. encoding of urls. As we accept IDN-domains, a small piece of code that handled the encoding that wasn’t very well written, which made the script emit errors and halt less than half way through.

That might explain why I haven’t seen as much traffic to certain pages as expected, hopefully the sitemap.xml will be re-indexed in a while and traffic returns.

There is also a humans.txt file available as well, with credits to the helpful people around me.

No responses yet

A few more ranges

Mar 22 2011

Added about 35 new IP-ranges, mostly media in the USA but also European and even Asian. The count is 1130 monitored ranges, adding more as they become available/found.

A nasty encoding bug I’ve been chasing for a couple of days has been resolved, it also prevented the pages from rendering valid XHTML 1.0 Strict results. Should be easy to up the pages to HTML5, some modifications necessary though.

No responses yet

Craptarded!

Mar 15 2011

Somehow I managed to fill up the server disk with either cached data or logs, probably cache, discovered it while at work and had to dig up my login credentials for the VPS hosting, added more disk and pressed “play”. Seems to work at the moment, but I’ll check what happened when I get home tonight.

The service did not go down, just the website .. while serving pages. In a little more than two years, I’ve had about 4 hours downtime.

No responses yet

Maintenance update

Mar 13 2011

Just updated a lot of code and templates of the CreeperFramework, preparing for further updates. No new ranges in this update but I’ve got serveral up my sleeve.

Some optimization, prettification, validation of HTML, tweaks on SEO-stuff, javascript and will turn on compression in the next update.

I’ll put some effort into completing the WordPress plugin, CreeperPing .. as the project has gone stale. Need to put this one out the door.

Have given evercookie‘s a thought ..

No responses yet

Mentions about MediaCreeper

Mar 08 2011

Reactions to MediaCreeper turns up every now and then, the latest I found were kind of speculative .. http://gertfrost.se/fa-kanner-till-mediacreeper (in Swedish) .. didn’t really take a stance for or against, just contemplating about the fact that it exists and thereabouts.

Silverback (also in Swedish) writes about MediaCreeper, commenting the fact that he can actually see that media circles his blog regularly.

No responses yet

API considered stable (enough)

Mar 07 2011

The MediaCreeper API (http://api.mediacreeper.com/) has been online for a couple of weeks now and are considered stable. It has been linked on the Swedish OpenData-site, a sort of repository of links to open APIs and data collections.

The MediaCreeper API is licensed under CreativeCommons Attribution-NonCommercial-ShareAlike or CC BY-NC-SA for short.

There are a non-public part of the API that isn’t completed yet, the CreeperPing. It’s a WordPress plugin that will interact with MediaCreeper over a Server-to-Server protocol, capable of circumventing blocking and blacklisting of CreeperFramework (MediaCreeper and variants).

No responses yet

Multilingual FTW?

Mar 07 2011

I just uploaded a PO file to crowdin.net and checked a few languages I consider “primary” as active for translation. Go to http://crowdin.net/project/mediacreeper-multilingual to help us make MediaCreeper truly multilingual. The base-language is English and if you know French, Spanish, Danish, German, Finnish or Dutch, please help us with the 72 text-fragments that need translations.

I’ll add the languages to MediaCreeper as they become completed.

Next round of translations may include .. (your language here). Suggestions welcome, just comment on this post.

No responses yet

« Newer posts Older posts »