Hugo Estrada's House of Wonder

This one takes a csv file and breaks it into files with 100K records, adding the header on each one of them.


#!/usr/bin/env ruby

fileString = ARGV[0]

if ARGV[1]
outputDirectory = ARGV[1]
else
outputDirectory = "output"
end

if !(File.directory? outputDirectory) then
Dir.mkdir(outputDirectory)
end

fileName = fileString.match(/(^.*)\./)[1]
extension = fileString.match(/\.(.*$)/)[1]
data = File.open(fileString)
header = data.readline

i = 1
fileIndex = 1
limit = 100000
output = nil

data.each { |line|
if i >= limit then
if defined? output then
output.close
end
i = 1
end

if i == 1 then
newFile = fileName + "#{fileIndex}." + extension
puts newFile
output = File.new(outputDirectory + "/" + newFile, "w")
output.puts(header)
fileIndex += 1
end
output.write(line)
i += 1
}
#output.close
puts "End of Script"