CASCON, DeveloperWorks, and half-year mark at IBM (part 2)

Yesterday, I attended CASCON 2009 (Center for Advanced Studies Conference). My team was hosting a workshop on the Technology Explorer, so I attended a part of that, as well as another workshop on Software Engineering 2.0 and Research 2.0 The latter made me realize the importance of being able to pool data from various resources, and link them to one another appropriately, so that we can make useful conclusions and findings. The workshop also talked about how to filter such wealth of information so that we can easily identify what is important and what is not. For example, in an IBM software developing tool called Jazz, developers can be notified if changes to the code have been made which will affect the piece of code they are currently working on. They can also be notified if someone comments on their code, files a bug, etc. Team members can also put other team member names in a flag for a comment, indicating that it needs review by that team member, and indicate the level of urgency. However, the test-run of this product showed that users were getting too many notifications, which slowed down the productivity. They ended up ignoring everything eventually, including the critical notices, simply because they couldn't be bothered by the flood of pop-up notifications. The developers of Jazz solved this problem by creating an inbox, sorting notifications in order of urgency. This level of urgency is initially set by whoever created the notification, but it is also modifiable by the receiver, so that they can move tasks up and down the queue.

After attending that workshop, I reflected on the TE developing team. It's a small team, consisting of 4 people including myself. While Jazz sounds interesting as a tool, I think it would just be a lot of overhead when the team is that small. As it turns out, they were using Jazz at one point, but somewhere along the way, they stopped using it because they needed everything on SourceForge anyway (since it _is_ an open-source project after all), and it was just doubling the amount the work/commenting involved. I do think we could use some sort of rss feed so that we are more up-to-date on what changes have been made to the TE on a more regular basis.

Lastly, my article on a DB2 plug-in support got published on the IBM DeveloperWorks site! I wrote a module for TE that supports a security plug-in written for DB2. The plug-in was written by Gene Kligerman, and is called "db2auth." In a nut shell, it allows for authentication to DB2 by storing the information in a database in DB2 (i.e., DB2 doesn't need an external authentication source if you use this plug-in). Storing such information in the database obviously raises some security loss, but it could prove to be useful as a temporary step when migrating from other database software that also store authentication information within the database. The module I wrote for TE allows users to create/delete/modify users and groups with a graphical interface, as opposed to the command-line interface that the original plug-in provides. It provides four views of the data, as well as buttons to drill down to more detail about each user or group. The article has some screenshots, so please take a look :)

CASCON, DeveloperWorks, and half-year mark at IBM (part 1)

Where did the time go? It certainly doesn't feel like 4 months since my last blog entry...

So in the past 4 months, I've been working on multiple versions of process-based and thread-based workload drivers in C and Java for DB2. The drivers forked off or threaded off their workers to perform various queries, and reported back to the database how many read/write/total queries their workers performed per second. Their primary usage was to demonstrate what will be DB2v9.8's ability to balance workload, among other things.

As part of the DB2 + system performance monitoring, I also wrote some perl scripts to write results of various system commands to the database. I think one of the hardest parts for both the perl script and C programs, was how to get them working properly on various distributions of Linux as well as AIX. Drivers didn't install the same way, they didn't run the same way, some modules just weren't supported, etc.

The demonstration was performed through the Technology Explorer, where I helped build the control panel to start/stop/modify the workload, as well as graph the number of transactions(queries), for a visual representation of the data for clients. My team made videos of the various scenarios to exhibit at IOD (Information on Demand) at Las Vegas last week, which was quite a success.

I've also been working on setting up a Linux server for our team, which will serve as a SVN server, web server, Trac server, and team database (DB2) server. Prior to coming to IBM, I'd had absolutely zero experience with any kind of server, so this was a new and interesting experience for me. I checked out 3 books from the library, pored over pages and pages through the internet, learning what the necessary parts were, how to install them, how to configure them, and why they weren't working. It's difficult finding the right answer when there are so many linux distros out there, but I feel I've learned a lot about Linux in general, and just today, I helped someone set up their Trac server to work with IBM's LDAP. The server set-up is almost done, and I'm now writing tutorials and scripts to make maintenance easier for both users and the administrator.

Exactly 2 months at IBM

Wow I don't even know where to start; it has been so long since the last update. This will probably be a long entry. Exactly 2 months ago, my internship at IBM started with a half-day orientation session, followed by a few days of administrative issues and setting up of my workstation. I joined right when IBM was testing for its release of DB2 ver. 9.7 and TE (Technology Explorer) ver. 3.5.

For the first few weeks, my main task was to get familiar with TE and how it works with/connects to DB2, as well as picking up PHP. To be honest, it felt like I was getting nowhere, although my team mates assured me that I was making progress. At the end of May, I had to take a week off to fly back to Japan for a friend's wedding, and when I came back to work, it was already time for regression testing of TE 3.5.

The purpose of the tests were to ensure compatibility with DB2 ver. 9.7 as well as previous releases of DB2, compatibility with various browsers (Firefox, Internet Explorer, Chrome, Opera, etc.), and that features available in the previous TE version still worked in 3.5. During testing, numerous bugs were reported, and as I helped find some of them, I finally felt that I was actually contributing something to the team.

After regression testing, I also helped with what we call "dynamic columns" a.k.a. "delta columns" in TE 3.5. These columns are configurable by the user so that TE can perform min/max/average type calculations on any column value over time intervals (e.g. number of reads per second). My goal was to create a GUI test on TE to see that these columns were being calculated properly, as well as fix (with MUCH help of my team mate -- so much, that my manager and connection coach joked about extending a workstation at his cubicle).

On June 19, TE 3.5 was released according to plan on SourceForge, without any visible rushing to my surprise, unlike submissions of school group projects (haha). There were Timbits for celebration (we also had a team BBQ which I missed because of Future Blue Day, an activities day for interns, which was FUN and I made some new friends, but cost turning 3 shades darker despite multiple layers of sunscreen), and my team mate dropped by to inform me that my name was in the credits as "owner of the dynamic columns." It never even occurred to me that my name would be publicized in an official IBM product. Flattering, but also scary!!

Since the release, I've been studying for the DB2 v.9 Fundamentals Certification Exam and working on a new project for TE/DB2. This time it's in C. Working with this team involves dealing with many languages (JavaScript, PHP, XML, HTML, SQL, and C) and features, and I like the diversity of my tasks. I'm also making use of a lot of what I learned in the past year or so of classes (mostly csc209, csc369, csc301, and csc343). The SQL knowledge from csc343 (Databases course) definitely came in handy during the Certification Exam. In fact, I just took it 7 hours ago, and I am now DB2 v.9 certified! :)