PDA

View Full Version : Wordlist Help needed



7hr08ik
2013-04-09, 22:23
Hey guys,

I think im posting this in the right section

I have a little problem i need some help with. I have several wordlists that i use for my WPA testing. I start with a small list of common words. Then i move on to a bigger list ive compiled using purehates, naxtoe, etc. and then if im really bored i move onto a huge list of words. But what if im wasting time by checking the same words?

How can i compare the lists and remove the duplicates without combining the files.

I have my first list...Common.lst
Then my next list...Hackers.lst
Then the huge list ..Huge.lst

I want all the words in Common to be removed from Hackers.lst
I want all the words from Hackers.lst to be removed from Huge.lst

You see where im going with this.

I think its possible to do this with grep commands but i cant figure it out. Ive done some googling and found a couple of commands but i cant get them to work. What should i be doing to get what i want?

Hakuza
2013-04-10, 12:24
This is easy.

# man sort
# man comm

Quick example:


rtm@kali:~/temp$ cat list1.txt
alpha
bravo
delta
echo
rtm@kali:~/temp$ cat list2.txt
bravo
charlie
foxtrot
rtm@kali:~/temp$ sort list1.txt
alpha
bravo
delta
echo
rtm@kali:~/temp$ comm list1.txt list2.txt
alpha
bravo

charlie

delta
echo
foxtrot

TAPE
2013-04-10, 21:08
http://www.backtrack-linux.org/forums/showthread.php?t=38522&p=191268&highlight=#post191268


Old post, but still should help you on your way ;)

nix
2013-04-11, 13:53
i have a fiew word list from small to masive so i created one big file and to make sure that there are no duplicate i did the following

sort -u * >> mywordlist.dic

sort -u will sort them and make sure they are unique

hopfully this helps

Cheers

7hr08ik
2013-04-11, 16:11
Well between the time of me posting this and checking it today. I spent a bit more time searching and managed to find a few things. After testing them out i`ve found that the `comm` command is the way to go.

comm -2 -3 /root/big.lst /root/small.lst > /root/big-sorted.lst

With this i can remove all words in the small.lst from the big.lst. That way i can start aircracking with the smaller lists and work my way up without re-checking the same words

Ive just gotta hope that comm can handle my 13Gb wordlist

Malachai
2013-04-15, 03:17
Well share your 13 gig list buddy ...lol... I have the same issue trying to combined small middle and large file together but take out the duplicates from each other.

thorin
2013-04-16, 17:02
You should look into the uniq command:
http://www.computerhope.com/unix/uuniq.htm

You should also checkout TAPE's blog, he has tonnes of posts on wordlist generation and manipulation:
http://adaywithtape.blogspot.ca/
(Posting archive list is towards the bottom of the rightnav.)

n1tr0g3n
2013-04-21, 00:11
Yeah TAPE's site definately is the one to check out for this, he even has a script that can help you clean up your wordlist WLM (Wordlist Manipulator)

http://code.google.com/p/wordlist-manipulator/

TAPE
2013-04-21, 13:16
The WLM script was originally based on Purehate's wordlist script, and grew from there.

When 'cleaning up' a wordlist a lot of temporary files are/were created. I tried to limit
that in the latest release, but still, if you are dealing with a huge wordlist in the GBs..
then that could cause problems with the temporary folder if you are running the script
on 1 disk and planning to save on another.. of course depending on your disk space available..

TH3CR4CK3R
2013-04-26, 01:04
haha, thats cool. All my homies are on here :cool:

The latest release of WLM is awesome when it comes to the 7.1 option, it is wayyy faster now!!

eHack
2014-07-07, 16:39
create your wordlist : crunch [min] [max] -o /root/blablabla :v
i think, i am a newbie :(

mmusket33
2014-07-08, 12:18
You can get a very good WPA word list thru torrents. Go to the pirate bay and look for the below listed wordlist.

WPA-PSK WORDLIST 3 Final (13 GB).rar

Furthermore if you search "WPA Wordlist" you will see other choices but this is the latest and to our knowledge the best. There are no words less then 8 characters in length and duplicates have been removed. We run it with elcomsoft and a few mutation choices like "case" and get hits regularly thru this list

MTB