Thursday, August 25, 2011

Process big text files in Python.

I'm pretty sure that you could do this in several different and better ways(as usual when it comes to coding). But this is my solution for reading big text files in Python.


from sys import argv

file = argv[1]
input = ""

inputData = open(file)

#Function for reading chuncks

def readChuncks():
return inputData.read(1024)

#Iterate through the file

for data in iter(readChuncks, ''):

input += data #process the data

inputData.close()



Any thoughts/comments/suggestions would be greatly appreciated :-)

No comments:

Post a Comment