Hakidashimado

Adventures in data continue. Some time ago, I wrote a program to take data files from an instrument and plot/reduce/summarize the pre and post test storage moduli, and the creep test. It took a while, but I finally worked through all the issues (including an invisible ASCII form feed character that made the Python code trip out. Srsly, who uses form feeds anymore? Those haven’t been used since punch cards and line printers!). Along the way, I worked out how to generate a matrix of graphs for large-amplitude oscillatory shear, if I ever get to do that. A girl can dream!

This years entertainment is to take data files from 2 different mechanical test systems and slice and dice the data for plots, and modulus of elasticity. Of course, they have different formats. Of course they don’t even name themselves the same. And why would I even expect to be able to have the same code for each of them?

It occurred to me that maybe, I don’t know, other people have had this problem, and I could ride the coattails of somebody smarter, or lazier, than me. Because we all know the characteristics of a programmer are laziness, hubris, and impatience (thanks, Larry Wall!). So I looked for python parsing data files and found https://github.com/webmaven/python-parsing-tools And then I picked out http://pyparsing.wikispaces.com/ just because the description didn’t have too many strange words, and it looked like it had recent updates, AND a lot of other questions about parsing mentioned it.

So far, I have spent 2 days on it and I have managed to feed it a comma separated line and get back floats! And toss a file at it and extract parameters from quoted strings. And possibly understand how to feed it a line with tab-separated numbers and get them back as floats. This is progress, but is it fast enough progress? And it looks like tab delimited and comma delimited data lines are going to be a challenge to put in a one-size-fits-all parsing solution.

I need a beer…