I removed the page since it hadn’t been updated since 2010 – nothing much written about MediaCreeper at the moment, it has just been removed visually, it’s still there if something would happen.
Yeah, about that…
I’ve started to code on it again, after being busy with lots of other stuff in my life, like having a son, getting married .. that sort of stuff.
The core parts are pretty much done, working on some authentication parts now and adapting/adjusting the plugin with the rewamped API.
In short; media houses will not able to block traffic to MediaCreeper – as some actually are doing to avoid showing up on lists on MediaCreeper. A visit from a media house on a blog where the CreeperPing plugin is installed, will be transferred directly from the blog-hosting server to MediaCreeper, without passing the media house so there is nothing for them to block or filter.
A few selected friends will be beta-testing after the holidays, or whenever I had the time to complete the code and internal testing.
I’ve started toying around with yet another incarnation of MediaCreeper, to conserve bandwidth, make it fit better on mobile devices and to learning new technology.
Working on the internals at the moment, updating the API to handle requests from the HTML (HyperText Markup Language) 5 frontend.
More details later.
MediaCreeper were somewhat broken the last couple of days due to a crashed table in MySQL, which in turn made the logging functions freak out and no referrers were saved, hence the amounts of “(unknown/hidden referrer)”. Also causing some errors to show as joins with broken tables and indexes ate a lot of memory.
Table repaired, memory added and a few other tweaks, should hold up now as there seem to be a lot more traffic, as there has been a WordPress plugin released that accesses the API to gather data and display in the WP Admin interface. Would also be great if the plugin could add a reference to the tracking image somewhere so the blogs using it also could be found in the logs…
This pushes some issues for further development up the todo list; MongoDb and minimize the memory footprint for the code base .. maybe even prepare/adapt for cloud-based hosting.
After a long period of inactivity, at least visible inactivity, we have got a bunch of new ranges and are also starting a round of evaluation of old ranges.
Work has started to clean up code, to prepare release of CreeperFramework as open source on github. Further ahead the database layer is gonna get an overhaul, a switch from MySQL to mongodb.
An evaluation of the usage of the API-site to determine if it’s worth the effort to convert it to node.js or not.
We had a Facebook Group, https://www.facebook.com/groups/45274699051/, which were emptied when Facebook decided to change format/layout/whatever .. a few people found their way back to the group.
As the Facebook Pages is a bit more user friendly, like in “just click [Like] to join”, we started one at https://www.facebook.com/MediaCreeper.
I got a message from a friend on twitter the other day, about the WordPress plugin I was developing a while ago. Up to now, it has stalled, completely, as the development of the API-site/service of the CreeperFramework went high priority the plugin were left behind, none of the API-calls that made it tick are the same, so I’ll have to rewrite those parts.
The CreeperPing plugin for WordPress is a simple way to evade filtering and blocking that some media houses do, as communication happens server-to-server, then they can block MediaCreeper all they want, their trails will still be visible..
I’ve never written a complete WordPress plugin from scratch, only hacked up existing ones beyond recognition, this will be an interesting journey. I’ve got professional WP-developers to hold my hand while coding and hopefully it’ll be done in the near future, a few weeks at most.
Found this blogpost on G+, Big brother now spies on journalists, keeps records of online activity which led to DHS: Privacy Investigations & Compliance Reviews, could not let this go without atleast a comment.
In the US, it’s now ‘OK’ to spy on journalists online. Yup, the journalists is now lumped together with terrorists and organized crime.
When I started MediaCreeper, it were aimed at journalists at big media-houses in Sweden, as a reaction on lack of reaction from the ‘fourth estate’ and individual journalists when repressive laws were about to be signed. Since then it’s been up over 1200 days on the internet.
I’m all for journalism and investigative work/groundwork for deep and analytical articles, which often require people inside companies, personnel in governmental settings to be able to speak with journalists without reprisals — informants who step forward in short.
This development in the USA is really disturbing, the USA lowers itself to the level of dictator-led states where the media is controlled and censored or completely taken over and used as a propaganda apparatus. This is bad, really bad.
After last incident I’ve been keeping an eye on the server, the traffic generates huge log files which almost filled up the disk, even when rotating them each day…
So, all logging has been turned off completely .. not that it was ever used for anything else than checking the flow of requests .. but ~4Gb logs aren’t practical to handle. All the Apache logs are gone, hello disk space!
Hopefully, the server will be somewhat more stable from now on ..
This evening I browsed some sites and noticed they had the MediaCreeper badge on them, I clicked on them and came to a completely blank page, WTH?! Tried to reload the page, no success.
Fired up SSH to connect to the server, logged in and found the disk full. Stupid mistake, I know.
When I configured the server back in 2010 after moving it to a separate, dedicated VPS, I figured it’d only require about 20Gb of disk — I was wrong.
I nuked a few of the logfiles as they are not needed for every day use and freed up some space, tweaked logrotate a bit and still had > 60% disk usage. Added some more disk and holding thumbs.
MediaCreeper shuffles ~50Gb/month at the moment, I moved it off my other hosting when it touched 20Gb/month .. It have more than doubled in traffic since 2010, popular service eh?
Now it should run without filling up the disk for a while, unless it touches 70Gb/month or in that area..