Some minutes ago, I saw an interesting tweet from Mikko H. Hypponen saying that he found out that all (yes, as in ALL – 35,513,445 ) google profiles addresses can be retrieved from a single XML file . Looked through it and , yeap, he was quite right.
Well , all these information is going to be useful somehow ,right? Right. In case it’s going to be removed here is a simple way to harvest them before that happens :
#!/usr/bin/env python import urllib from BeautifulSoup import BeautifulStoneSoup as bs xml = bs(urllib.urlopen('http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xml').read()) for i in xml.findAll('loc'): try: urllib.urlretrieve(i.text, i.text[35:]) print 'Downloaded %s' % i.text[35:] except Exception, err: print '%s could not be retrieved' % i.text print 'All done'
That’s it, save it , run it and wait 🙂 Not that I used it, but I calculate that you get around 1.7 GB worth of profile links .
Well , the juicy part is obviously the harvesting of the information from the profiles themselves. People are mentioning on twitter that Google is aware for a long time, or at least should be. Thoughts about the potential implications from that harvesting, on a blogpost to come .
Yes, WEP is deprecated , proved to have some really nasty flaws , old story right ? Searching for “wep crack” in Google brings some 1 million results.A very nice explanation of why WEP is insecure can be found in this book . There is really no interest in discussing the matter anymore.
The interesting part of the story is how many people are still using WEP despite all that. Reasons vary from users ignorance to vendors’ continuing support of WEP in home routers/AP products e.t.c . Moving around the last 2 years I was always surprised by how many WEP – “protected” wireless LANs one can find in every city/neighborhood. The days when cracking into those was a challenging process are long behind , what I find interesting is having an overview of the amount of WEP usage in any given circumstance. Hence I came up with the following python script that takes advantage of the iwlist utility from the wireless-tools package and provides an overview of the usage of WPA/WPA2 and WEP in the wireless LANs that are detected.
It needs the pyparsing module (python-parsing package in debian/ubuntu)
#!/usr/bin/env python import subprocess from pyparsing import Literal, SkipTo, OneOrMore def wepPercentage(output): last_line_a = Literal('Group')+SkipTo(Literal('\n').suppress()) last_line_b = Literal('Authentication')+SkipTo(Literal('\n').suppress()) start = Literal('Cell')+SkipTo(Literal('\n').suppress()) end =OneOrMore(Literal('IE: ')+SkipTo(Literal('\n').suppress()))+~last_line_a|(last_line_b+~Literal('IE:')) line = ~end + SkipTo(Literal('\n').suppress()) expr = start+OneOrMore(line)+end wpa_count = 0 wep_count = 0 unenc_count = 0 for l in expr.searchString(output).asList(): li = ''.join(l) if 'Encryption key:on' in l: if 'WPA' in li: wpa_count += 1 else: wep_count += 1 else: unenc_count += 1 tot = wpa_count+wep_count+unenc_count print "WPA / WPA2 :%d Percentage : %d %% " % (wpa_count, (wpa_count*100/tot)) print "WEP :%d Percentage : %d %% " % (wep_count, (wep_count*100/tot)) print "Unencrypted :%d Percentage : %d %% " % (unenc_count, (unenc_count*100/tot)) if __name__ == '__main__': process = subprocess.Popen('iwlist wlan0 scan', shell=True, stdout=subprocess.PIPE) output = process.communicate() wepPercentage(output)
I live in a rather densely populated student dorm in Stockholm and at the time my output is something like :
WPA / WPA2 :28 Percentage : 82 %
WEP :6 Percentage : 17 %
Unencrypted :0 Percentage : 0 %
almost 20% of all people here that use some kind of wireless router , chose WEP . Great 🙂
How about your neighborhood ? Feel free to share percentages .
ps. It would probably be easier to use python’s re module instead of pyparsing . But I needed to use pyparsing for a project so I thought i d get my hands dirty with it anyhow.
ps2. It can probably be done in 5-6 lines in a bash shell using awk too .