This may seem obvious to most, especially to regular blog readers but don’t forget the development community when working on projects. There is a large community of developers and many are willing to share ideas, code, and techniques.

When I am stumped, or simply taxed with a problem that has a common element to it, I have a pretty standard sequence of steps I take to see if I can get help or find a component that someone else created already. Most projects break down into 2 items, common problems and issues specific to your company, environment, etc…

Recently while talking with our operations manager he mentioned he was having problems with an old isapi filter we use to receive and parse incoming customer service tickets. The filter was written over 6 years ago, and the source code is gone. We are lacking a good deploy plan for this filter and we needed to replace it soon. We came up with a very simple solution. Have our mail server dump the incoming .msg files to a secure directory, and write a simple parser to monitor, parse, and insert the tickets into our system.

So I set to work. We are primarily a ColdFusion shop here, however we do write in many other languages. This was a task that would be easily written in ColdFusion and would also run extremely fast. It seemed like the right tool for the job. ColdFusion also has an event monitor that will monitor a directory for new or changed files and trigger code. I have had mixed results with it in the past, but this seemed like a good place to try again. If not, I could simply schedule the script to run at set intervals.

First step, open a msg file and see what we got. Wow, that was going to be a mess to parse. But I had a hunch someone wrote a basic email parser in ColdFusion. If I found one, that would let me spend the time coding the specific pieces of this project related to our environment. So I immediately turned to the community.

Here is a general run down of the steps I take when looking for some code or tips.
- I send an email to my development team. If they already wrote or found similar code, then problem solved.

- Next, I Google my issue. In this case “coldfusion email parser”

- Check a few of the code community sites I have bookmarked in Delicious. In Delicious I keep code and community sites tagged with the language they serve. In this case, fusion authority, adobe, ray camdem’s blog, etc… Whenever I come across a code repository, or good language specific blog, I tag it in delicious. You never know when you’ll need it.

- Twitter: I don’t send a tweet often to ask for help rather, I run a search similar to Google in TweetDeck. I may create one or two searches and let them run. This usually leads me to people who are having similar issues, or possibly written about something I need.

- Finally, if I have time to wait for a response, I go back to some of the code specifics resources I mentioned earlier, and post a question.

One of these usually leads me to the path I need. I have had great success with these methods. Sometimes the module I find is slightly out of date or missing a few key elements I have to add. This is fine, it at least give you a starting point. For example I found a BaseCamp component that worked with their old API. I found it quite easy to add the new API methods and bring it up to date. It provided a nice clean framework to work with. I recommend if you do make changes that the community might find beneficial, please give it back to the community. As soon as I clean up the BaseCamp component, I plan to submit it back to the repository I found it.

This same process can be applied to whole open source apps, you are not limited to simple components when looking for a building block for your project. That’s what makes the development community on the web so useful.

Do you have other techniques, or processes to find help and code you need? Let us know in the comments.

Update 5/7/2009 by Bill: I did eventually find a ColdFusion component that parses emails via my google search. It was on Tom Mollerus’ WebLog. It worked great, and saved me a lot of time. I was able to get this running in a few hours, instead of days.

Related posts:

1. Using the Basecamp API to Create Project Reports A few months ago I wrote a post describing the...
2. good development teams share information Sometimes its hard to effectively share everything among all members...
3. Software: 6 Questions to Ask Yourself When Deciding to Build or Buy A common dilemma of many tech managers and businesses in...

Related posts brought to you by Yet Another Related Posts Plugin.

Solid Architecture is a Dying Art

When I was in my CS program from 1992-97, we coded in ANSI C and COBOL. As a web developer, a lot of the specific code we wrote is no longer applicable today.  One major concept that was vital to writing code during my college years that I feel is useful and probably not a focus these days is writing tight, effective code.

When I graduated college, my first job was pretty groundbreaking. We were writing full featured mobile apps for field use. We had to work with heavy handheld devices that communicated with CDPD or dial-up to transfer their data. They had 256k RAM, 4Mb harddrives, and 9,600 baud modems. We had to focus on writing tight code, and conserving space on locally and for data transfer.

Today harddrive space, RAM, and processing power is cheap. Very cheap, and you don’t even have to have your own. You can rent power and spawn processes and database space as needed on Amazon ec2 servers.

Even users’ machines are way more powerful. Users have high speed connections with unlimited data and huge hard drives. Because of Windows Vista, even non-gamers are putting 4Gb RAM in their machines. This is great for us as developers in some ways. Running slick Flash and AJAX apps on clients machine is no longer a problem. We can be very creative, and deal with problems later as they arise.

Too often traditional computer science techniques are being thrown out the window. This is tolerated because of the cheap power we can buy. A shortfall in the code or architecture is simply fixed by adding hardware. Twitter blew up so fast their initial poor design had to be tweaked many times. To this day, they have some questionable design elements in their architecture and it shows. Fail Whales and slowdowns are part of the daily routine of twitter users.

I am not saying we can’t or shouldn’t take advantage of cheap hardware these days. I’m also not implying that the developers at Twitter are not smart, they certainly are. Spending hundreds of hours coding for problems you don’t yet have is a huge waste of time. But a good artecture basis is imperative, so that changes can be made with minimal impact to users.

This is definitely a dying art, but its importance will become more evident as mobile computing becomes more relevant. More and more users are browsing sites on smart phones and net books. These are less powerful appliances by nature, and are usually coupled by slower 3G data transfer speeds. That means bandwidth is now becoming slower and more expensive again. Storage becomes a premium again. Developers will need to be more mindful of their bandwidth and storage usage when writing code.

Finally I want to briefly mention a story I was talking about with a colleague today. In 2006, users were racking up huge data fees downloading mobile maps. Google assigned a room of their best engineers to the issue, and could not come up with anything. Then one young designer came up with the idea to simply reduce the amount of colors on the maps from 256 to 20-40. Users didn’t notice, and amount of binary data to transfer was greatly reduced. It was heralded as being genius and a breakthrough idea. Granted it is smart, but do you honestly think a room of old school DOS or UNIX developers would not have thought of this sooner? Tricks like this have been around forever.

What do you think? Should you code away at your users expense, or be more mindful of conserving resources? And if you are in school or recently graduated, tell me if I am completely wrong and maybe you were taught to focus on good design.

Related posts:

1. Use Project Collaboration Software We use BaseCamp here for managing projects. It is extremely...
2. Knowing your users will help make effective decisions If you are reading my blog, there is a very...
3. Link: The Duct Tape Programmer This is a great Article I found on Joel On...

Related posts brought to you by Yet Another Related Posts Plugin.

Semi-weekly wrap up of links, tidbits, and other sites that came across my plate.   Code, development, management, anything I found interesting since the last post.  Feel free to suggest other links in the comments.

Web Tools

• 15 Helpful In-Browser Web Development Tools – Most people are going to be familiar with these tools, especially if you found this blog, but nonetheless, its a good list to keep handy. Even if you keep the list to help new junior developers setup their browsers for development.
• Free Web Resources – I only recently bookmarked this site, and follow it briefly. So far what i’ve seen has been borderline. I will add it to my RSS reader and give it a few weeks. Does anyone read this site?

Web Talk/Blogs

• The Most Powerful Word is No – Short message from 37Signals about saying NO.
• How to Be a Program Manager – Good article about a program manager and why good software teams need one. I would refer to this role as a Product Manager, and I plan to post my thoughts on Product Vs Project Managers roles in Web Development soon.
• Web Development is Moving On – Are you? – I really like this article. Talks about the changes in Web Development, and how we need to change with it to remain relevant. I recommend reading this article.

Web Tips/Resources

• Top 5 Sites To Learn Some CSS Programming – Good list of some intro sites for beginner CSS.
• 9 Common Usability Blunders – Anyone developing web software should keep these basics in mind when designing UIs.

Post any good resources or links you use in the comments.

Related posts:

1. Web Tools Tips and Talk: #1 Today, I am going to post a little weekly wrap...
2. Web Tools Tips and Talk #4 As usual, a semi-weekly wrap up of links, tidbits, and...
3. Web Tools Tips and Talk: #9 It’s been another 2 weeks (plus holiday weekend), so here...

Related posts brought to you by Yet Another Related Posts Plugin.

Here at SmartMoney, we have been using iterative development techniques for years. Even before the term was made popular in the technology field, we ditched the long drawn out style of project development. For the most part, we now employ a more cyclical style of development where individual features are coded, tested, deployed. New features are added based on lessons learned from the earlier cycle. From there we try to gauge the public’s reception to the feature through usability testing and feedback, then either work to improve that feature, or begin work on additional features within the same short cycle.

In the past we tried creating wikis for our projects and writing document specs. These are really a waste of time, and just do not work. They take weeks to create, and are often outdated as soon as they are delivered.  Better yet, even if they are not outdated, they are usually ignored. Iterative development gives you the opportunity to deploy, test, and most importantly to be wrong.  If a feature is not accepted by your users, you can easily refine or remove it.  You are more apt to admit your mistakes and remove a feature that took 2 weeks to deploy, than one that took 6 months.

The problem with getting buy in for iterative development at most companies is usually not with the technology group. Developers and technology managers usually embrace the techniques. It keeps things fresh, it involves less red tape and lets developers get to what they love – code. I find for me it keeps things moving. You feel less stale when you are always moving through these mini life cycles.

I find resistance to this style of development tends to come from the business side. Groups like marketing and sales prefer a larger “completely finished” project to sell to clients. Project managers tend to prefer larger projects to maintain their value on the project. The larger the project, the more management it will need. Finally, some people just don’t consider it a web project without a large cycle of gathering specs, creating wireframes, refining code, QA/Testing, repeat ad nauseum, then finally deploy. These are the people in your company that need to be sold on the benefits of Iterative development. I found an article today that discusses getting business people and anyone who needs it to embrace the philosophies of Iterative Development.
Embracing an Iterative Approach to Site Development. Its a quick read, and simply states some cost and time benefits you may find useful when trying to sell the idea to someone where you work.

Iterative development is practical and effective because it ensures you are keeping your projects lean and agile.  I really recommend looking into this style of development and trying to find areas where you can incorporate it into your workplace.  Even if the majority of your projects employ long spec-code-test-deploy cycles, you may be able to work with these methodologies in small ways.  Try to take a large project and see what can be broken down into manageable iterative chunks. My experience is that you will most likely have luck looking at the back-end development needed. This area of a project has the lowest visibility, as most project stakeholders care about the UI. Get familiar with breaking down the requirements to a bare minimum, coding for that minimum, then repeat the cycle if necessary. Once you have a few wins, or successful iterative projects, you may find others in your company are more willing to adapt their style as well. The benefits will speak for themselves.

Related posts:

1. deployment schedules are not effective Simply put, deploying code on a set schedule is not...
2. Knowing your users will help make effective decisions If you are reading my blog, there is a very...
3. Use Project Collaboration Software We use BaseCamp here for managing projects. It is extremely...

Related posts brought to you by Yet Another Related Posts Plugin.

Sometimes its hard to effectively share everything among all members of a group.  We have weekly meetings to review our projects and their status and we use project collaboration software but sometimes we simply get absorbed in our own projects and workload.

A great way to share information among team members is through fixing error logs.  For years, our developers were responsible for monitoring and fixing errors on their projects whenever they show up in the error logs.  A year ago I implemented a new system for fixing errors which keeps developers working on new code, and aware of other projects within the group.

We have a team of 5 web developers.  There is an automated nightly process that gathers our application server error logs and distributes them to the development group.  In the past each developer was tasked to look through the logs for errors pertaining to their code or areas of responsibility, and fix them.  This was boring and inefficient.  At times all 5 developers were sifting through the exact same logs, looking at the exact same errors.

Under the new system, the logs are assigned to one developer for the week.  First, they look for errors new projects deployed in the last week or two.  These errors are assigned to the developer or team responsible for that project using bug tracking software.  We use an open source solution written in ColdFusion by Ray Camden called LightHouse Pro.  It works great for us.

With those obvious errors out of the way the developer now looks for patterns in the error logs.  These patterns could be a reoccurring error across the servers, or one server prone to a specific issue.  They may also notice a pattern happening at a certain time of day.  This is one of the many benefits of having the same set of eyes assigned to the error logs for 5 business days in a row.  These errors are addressed first.  If fixing one piece of code can stop a pattern causing 100s of errors, this is your biggest bang for the buck, and most worth the time and effort.  Finally they look through remaining issues and try to work on something they are not familiar with.   A developer is expected to devote about 1.5 – 2 hours each morning of their assigned week.  On Friday, they issue a summary of errors fixed, assigned, and the remaining open errors.

Sometimes when someone in the group is so close to fixing an error, or has invested time and research to that issue, they keep ownership of that issue.  This is not required, but the system fosters a sense of ownership and pride in fixing a problem.

Before we started this system, our logs were a mess and considered a chore to work on.  When we began, our logs could have 1000+ errors in a 24 hour period, and take a full 2 hours each day.  One year later, our logs average a few hundred errors, and usually take an hour or less to work on.  Some weeks, there are only a handful of issues to fix.

The are many benefits to this system.  Everyone gets familiar with various areas of the site they would not normally have worked on.  Developers get familiar with other pieces of code on the site, and get a better sense of the big picture.  We are able to spot error patterns and also get a chance to work with well written code.  All this allows us to develop more effectively on our own projects.

Related posts:

1. Allow Developers To Make Mistakes Keep an environment where developers, and all employees for that...
2. deployment schedules are not effective Simply put, deploying code on a set schedule is not...
3. Keep Teams Lean Developing effectively starts with the team. Of course projects requirements,...

Related posts brought to you by Yet Another Related Posts Plugin.