-
Wordlist Help needed
Hey guys,
I think im posting this in the right section
I have a little problem i need some help with. I have several wordlists that i use for my WPA testing. I start with a small list of common words. Then i move on to a bigger list ive compiled using purehates, naxtoe, etc. and then if im really bored i move onto a huge list of words. But what if im wasting time by checking the same words?
How can i compare the lists and remove the duplicates without combining the files.
I have my first list...Common.lst
Then my next list...Hackers.lst
Then the huge list ..Huge.lst
I want all the words in Common to be removed from Hackers.lst
I want all the words from Hackers.lst to be removed from Huge.lst
You see where im going with this.
I think its possible to do this with grep commands but i cant figure it out. Ive done some googling and found a couple of commands but i cant get them to work. What should i be doing to get what i want?
-
This is easy.
# man sort
# man comm
Quick example:
Code:
rtm@kali:~/temp$ cat list1.txt
alpha
bravo
delta
echo
rtm@kali:~/temp$ cat list2.txt
bravo
charlie
foxtrot
rtm@kali:~/temp$ sort list1.txt
alpha
bravo
delta
echo
rtm@kali:~/temp$ comm list1.txt list2.txt
alphabravo
charlie
delta
echofoxtrot
-
-
i have a fiew word list from small to masive so i created one big file and to make sure that there are no duplicate i did the following
sort -u * >> mywordlist.dic
sort -u will sort them and make sure they are unique
hopfully this helps
Cheers
-
Well between the time of me posting this and checking it today. I spent a bit more time searching and managed to find a few things. After testing them out i`ve found that the `comm` command is the way to go.
comm -2 -3 /root/big.lst /root/small.lst > /root/big-sorted.lst
With this i can remove all words in the small.lst from the big.lst. That way i can start aircracking with the smaller lists and work my way up without re-checking the same words
Ive just gotta hope that comm can handle my 13Gb wordlist
-
Well share your 13 gig list buddy ...lol... I have the same issue trying to combined small middle and large file together but take out the duplicates from each other.
-
You should look into the uniq command:
http://www.computerhope.com/unix/uuniq.htm
You should also checkout TAPE's blog, he has tonnes of posts on wordlist generation and manipulation:
http://adaywithtape.blogspot.ca/
(Posting archive list is towards the bottom of the rightnav.)
-
Yeah TAPE's site definately is the one to check out for this, he even has a script that can help you clean up your wordlist WLM (Wordlist Manipulator)
http://code.google.com/p/wordlist-manipulator/
-
The WLM script was originally based on Purehate's wordlist script, and grew from there.
When 'cleaning up' a wordlist a lot of temporary files are/were created. I tried to limit
that in the latest release, but still, if you are dealing with a huge wordlist in the GBs..
then that could cause problems with the temporary folder if you are running the script
on 1 disk and planning to save on another.. of course depending on your disk space available..
-
haha, thats cool. All my homies are on here :cool:
The latest release of WLM is awesome when it comes to the 7.1 option, it is wayyy faster now!!
-
create your wordlist : crunch [min] [max] -o /root/blablabla :v
i think, i am a newbie :(
-
You can get a very good WPA word list thru torrents. Go to the pirate bay and look for the below listed wordlist.
WPA-PSK WORDLIST 3 Final (13 GB).rar
Furthermore if you search "WPA Wordlist" you will see other choices but this is the latest and to our knowledge the best. There are no words less then 8 characters in length and duplicates have been removed. We run it with elcomsoft and a few mutation choices like "case" and get hits regularly thru this list
MTB
