za3tar

I have wanted to do this for a while, and now that i am on winter break, i finally did.

So here is the deal: I wanted to create a graphical map of the relationship between all the blogs on Qwaider Planet. More specifically, i wanted to represent each blog with a dot on a map, and draw a line between two blogs if one of them reads the other or links to it. So the end result will be like one of those Internet maps, but localized only for QP.

First, and without further adieu, here is a high-level readership map of all the blogs on QP.

Click on the image for a larger version.

The process was almost entirely automatic. On a high level, i retrieved the most recent posts of all the bloggers via their RSS/Atom feeds. Then iterated over all of them and parsed out comments by other bloggers on QP and the links to them.

Unfortunately it was not that easy. There were many complications that i had to get around. However, i will leave that for another post (or via email to those who might be interested).

Anyway, the picture above of QP as a whole makes sense. There are many bloggers who heavily communicate with many other bloggers; these represent the black blob at the center. Also, there are a few fringe blogs that communicate with less people on QP; these represent the few spikes that only have a few lines (edges) to the other blogs (nodes).

I created a large detailed picture with all the blogs, but it was 90 MB in size, and it crashed the program when i tried to compress it. Here is a link to it if you would like to give it a shot.

I also created a cooler individualized set of graphs of which bloggers read a certain blog, and the relationship between them. So for example, here are a couple of graphs for the blogs that have been recently linked to from my blog and the relationship between them. Again, the edges (lines) represent links.

Click on the images for a larger version.

I also expanded that to show all the blogs that i have had any sort of interaction with. So not only those that are linked to from my blog, but also those on which i had posted a comment on, but i have no recent link or comment from them on my blog.

Click on the image for a larger version.

Finally, i created the readership graphs for many of you!   Check them out, i think they are pretty interesting. Here are the maps for Qwaider and Maioush, Hani, KJ, Garfan, Batoul, Nizar, Jad Madi, 360 East, Black Iris, 7aki Fadi, Hreega, Moey, Kinzi, Mimi Cooks, Jessyz. and Mayyasi. I hope you like them. Again, each graph shows which blogs have recently commented on your blog, and what are the relations between them. If you want your own graph, please let me know and i will generate it and send it to you.

A long time ago, when search engines were relatively primitive, the “meta keywords” section of your site/blog were used as a first heuristic for search engine optimization. All of us, of course, want search engines to find us; so when we created blogs back then, we would dump anything and everything that could be –even remotely– related to us or the topic of our blog into that meta keywords section (some blogging engines even asked you about that).

Nowadays, hardly any search engine looks at the meta keywords or meta description sections of your blog. So they are more or less useless (as far as i know). However, it is sometimes fun to look at these keywords to see what each blogger thinks of himself or his blog.

Just for reference, this is how the “meta keywords” tag would look like in an HTML file:

<meta name="keywords" content="keyword,another keyword ..." />

For a little entertainment, i checked out the meta keywords of some of the folks on QP. Not everybody had them, and mostly all who did included their names or misspells of their names. Here are some of the interesting keywords:

• AndFarAway: Jordan, Design, Art, Amman, Islam, Arab, Chat, Blog, Pictures … (ok this one kinda works out)
  
• KJ: UAE, Dubai, Moogle, Humor, Funny, Thoughts, Hani Obaid (huh ??), Sheep, Holidays, Adha, Turkey, Thanksgiving, Religion, Ambush, Grand Theft Auto, Lost and Found, Police, Eyelashes, James Bond, Nutcase, OCD, Psychology, Traffic, Women in Coma  (what ???), Photography, Blood, Scary, Video, Violence, Zombies …..(essentially KJ encompasses everything under the sun!! .. and then some! )

Now if KJ’s keywords didn’t catch you, then his meta description might lure you in:

• KJ’s meta description: Jar of Juice is your ultimate source for humor, randomness, and your occasional serious post! … (somebody is confident :-p … actually this one also kinda works out)

• Hani Obaid (who apparently also is a keyword for KJ): Jordan, Amman, Blog, Canada, Wordpress, Montreal, Blogger … (what ? no mention of KJ ?!! )
  
• Qwaider: Photography, Philosophy, Movies,  Jordan, Arab, Blog, Islam, Computers, Science and Technology … (ok this one also works out .. but what’s up with that “LutsToLove” email address in the meta text ?)
  
• Cinamon Zone: Animals, Arab World, Art, Blog about Jordan Day, Books and Literature, Children, Courage, Diaries, Fekra, Funny, Health, Science, Miscellaneous, Nakba, Palestine, Phobia, Pictures, Comments, Ramadan, Movies, Sports, Thoughts, Translation, TV, Videos, Women, Work, Zucchini … (also everything under the sun …. but what’s up with Zucchini as a keyword ?!! )

I only posted this for a little smile, no harm intended. I would have also posted my own meta keywords if i had them.

… so if you find anything broken, please let me know.

أنا مع قرفان، شلة المدونين العرب الأشاوس شكلهم زهقانين شوي . صراحتاً أنا حسي النكتي ماكل هوى مؤخراً، فما بعرف إزا كني الشخص المناسب لأُسأل عن نكته هلئ، ولكن جواباً للـ “تاج” تبعت قرفان، خذولكم هالنكتتين اللي سمعتهم لما كنت في الأردن الصيف الماضي:

- واحد مسطول دقوا على بابه دائرة الإحصاء وسألوه: (إنتَ من أهل البيت ؟) .. فرَد عليهم وقال: (لأ، أنا من كفار قريش).

- طفيلي بيقول لجماعته: صلوا وصوموا وزكوا، مش أروح عالجنة وألاقيها كلها كركية!

(ههههه .. طبعاً هلئ قرفان لما شاف بياختي حرم يسألني عن نكت من اليوم وطالع )

Unless you haven’t used the internet in the past 3 years or so, you are probably very well acquainted with CAPTCHAs already; they are these morphed letters or numbers you see on blogs and signup forms that are supposed to prevent spam. They are a product of research by the “human-based computation” / cryptography researcher Luis von Ahn (also here) at Carnegie Mellon University.

CAPTCHAs solve a simple problem; Spammers can write small applications that scan for input forms and fill out these forms automatically and repeatedly. If you run your own blog, you probably noticed this (spam actually isn’t written by people usually ). So the answer to this symptom is also simple, if spammers are using computer applications to automatically propagate spam, make part of the fill out form something that the computers can not recognize. So a CAPTCHA creates a sequence of characters and morphs it so that the program that is generating it itself can not read it (and hence other programs won’t be able to read it either), and puts it on the forms. We, being mostly human, are actually able to read and decode that morphed text, and thus by solving these little problems, we can assure the web application that is receiving this form, that we are indeed humans (not spam applications).

Needless to say, the idea of CAPTCHAs was a big hit. These things sprung all over the web and are now used everywhere. Recent studies put an estimate number of how many seconds does each average web surfers spend daily solving these CAPTCHAs. Although the amount is small for a single person, aggregating this amount over all internet users gives us a huge huge number of wasted “human cycles”. So, Luis decided to utilize that time.

As many of you know, there is a huge effort to digitize old books for which no electronic copies exist. Digitizing these books uses a process called OCR which is also provided by your average home scanner. Unfortunately OCR is a bit error prone and some words are not recognized correctly. However, given the awesome power of recognition that people poses, even though OCR programs might not be able to understand certain words, we can. But, who will be willing to sit in an office all day “recognizing” words ? This is a mundane job to say the least. Can we somehow outsource this job to the masses (without paying them ) ?

Introducing reCAPTCHA

Connecting the dots seems easy now, and a re-engineering of CAPTCHAs is made, and out comes reCAPTCHA (again by Luis and his team). A reCAPTCHA is just like your ordinary CAPTCHA, but instead of one morphed word you now have two. One of these words the server already knows the answer for, and the other one is unrecognizable by the OCR program. The thought is, if the entity filling out the form knows the answer for the thing that i already know, then it is highly likely that it also knows the answer for that other thing that i don’t know.

So, now every time you fill out one of these reCAPTCHA forms, (or comment on this blog ) you are helping digitize one word. If this spreads wide on the internet, we can have our old library digitized in no time.

Anyway, after this long introduction and motivation, i just would like to announce that i am adding reCAPTCHA to my blog. There is a ready wordpress plugin for it so i urge you all to add it if you can.