One of my chief joys in life, often to dismay of others, is working with data. Some members of my family (who shall remain nameless) would characterize this fixation as bordering on obsessive, others just call me a "geek" and leave it at that.
I don't care.
Give me a data set and I can play happily for hours. The sky gets dark. The dog entreats me over and over to please throw her sopping toy so she can go chase it. The cats walk between my hands and the keyboard and rub their bodies against my face.
But I don't see any of that when there's a computer screen in front of me and columns of data points to graph and ponder.
Nevertheless, even though I like analyzing data, I don't like to do repetitive things that look like a waste of time. Maybe it's a holdover from all the years of filling pipette boxes-by hand- or washing culture tubes, or doing minipreps. Whatever. Excessively cutting and pasting things into text boxes at web servers is not for me.
Also, I'm not a programmer, I'm a scientist. I answer a question and move on to the next thing. The last thing that I want to do is spend a week writing a program to answer a question, and then use it and lose it. Or even more horrifying, have someone else try to use it and ask me for help. I know the rule. Every line of code that you write is a line of code that you need to support. Writing more abandonware is not the path for me.
Admittedly, I can be totally blind and inclined to take certain things for granted. Like my car, I don't ask how it works, nor do I care. Consequently, I was able to work with my company's software for many years, and be completely oblivious to the workings under the hood.
Then, we put out a new product (iFinch) and all of a sudden, for the first time, I realized that I could use our software in a class I teach!
I could have my students enter information into iFinch by using the comment form in FinchTV and we could collect all kinds of data. It's so cool!
But it wasn't enough. Even though the data were stored for each individual record, we couldn't obtain all of the data for the whole set of sequences.
Naturally, I complained.
Why do you let us store this data if we can't get our hands on it?
And the software developers set me straight.
It's all there, they said, open your eyes and you'll see it.
Thank you, Obi Wan. I thought snarkily.
But they were absolutely right. And I had been absolutely clueless.
All of a sudden I started looking around, poking through the database schema, and praticing phrases like: "SELECT * from table."
I was stunned! How could I have missed this for so many years?
We store all kinds of things, things I've wanted for years, and data that I didn't even know I wanted, but I do, I do!
Shakespeare would have said:
There are far more things in heaven and databases, Horatio, than are displayed on your web pages.
Yes, sometimes I'm a slow learner. But my eyes have been opened and now that I understand the whole relational database thing, I've fallen in love with SQL*.
In case you're wondering:
SQL is pronounced sequel and it stands for = Structured Query Language, it can be pronounced a couple of ways and you can find some amusing discussion about this from Joel. We use SQL to pose questions to a relational database.
Perl = Practical extraction and report language
- Log in to post comments
Ain't data analysis grand?
Your next step should probably be to investigate database report writing software that will allow you to take data from multiple SQL queries, integrate them (even across databases), and report on them in many ways including automated production of charts and graphs. Reports are usually also updatable, so that when new data is added to the database they can be run again without changing/editing the report software.
There are several report writing programs available. A good example is the freeware, cross-platform Agata Report. http://en.wikipedia.org/wiki/Agata_Report
Thanks chezjake,
maybe I'll give it a try.
We could easily exchange lives and the personalities (regarding data) would adjust instantly.
By the way, I always said Es Que El (the letters) because I simply never knew it was "sequel" .... I can't say "sequel." I can only say Es Que El. Stuck with that.
Ultimately, SQL comes out of early database languages like Dbase. I did a lot of database analysis with Dbase, through version V, then learned Clipper, which is a programming language like Dbase but more powerful and faster. In other words, you could more or less run your Dbase code in Clipper, but then add more (and compile it).
Anyway, SQL has its roots, as far as I know, in those early dbase scripts. (maybe it goes back farther)
Now, this may not interst you, but it is pretty easy to use PHP and SQL together. For what you do this may have very limited applications, but if you have a certain way you like certain data queried and presented quick and dirty, you can do that pretty easily with PHP.
And of course, most higher level programming languages have an SQL library. For instance, in Python, you can do all those SQL commands but have the results placed into python data structures. Since you do Perl, you would probably prefer the Perl SQL interface, which I assume exists.
In fact, I think I'm going to select PERL and SQL tonight as my bedtime reading.
By the way, I like to call SQL, which uses by convention UPPER CASE LETTERS IN IT'S COMMANDS, LIKE:
CREATE VIEW TREE_MEASUREMENT (ID,TRANS,SPECIES,DBH);
... and so on ...
"Shouting Quite Loudly" language.
You know about stored procedures, right?
You don't have to use upper case letters. SQL works either way, you can shout or you can whisper. :-)
That is correct ... it is a convention rather than a requirement. I think it is a tradition from earlier dBase days.
The perl DBI module is quite helpful.
http://dbi.perl.org/
SQL is not always your friend.
http://xkcd.com/327/
That's funny! I love that comic strip.