Finding a needle in a haystack

Well, today I needed to debug an error on the server. I needed to search the logs to find what the error was. Finding the particular information that I needed was difficult simply because I wasn’t proficient in vim and linux commands, and, the log was more than 20GB of data..

As usual, I looked for my all-knowing friend google. One of the developers on stack overflow suggested using grep instead because searching large files using vim is slow. So I went to the folder containing the logs to grep the particular regex I was looking for into a text file.

cat myerrors.log | grep theRegexIWant >> outputMySearchResults.txt

From there, retrieve (copy) the id that is relevant to that particular error and search using vim. So heading back to the massive .log file

vim myerrors.log

# using vim to find the particular error

Perhaps there is a better way, but at least I managed to find what I wanted from the logs. Probably truncating the logs to days or size is a better option. 😀

Leave a comment

Your email address will not be published. Required fields are marked *