As the days count down until I jet off to Las Vegas and release v2.0 of Agnitio I’ve been thinking about the past year and the lessons/things I’ve learned since I first released Agnitio – my first open source project.
Whilst Agnitio v1.0 was only released 7 months ago I announced that I was writing the tool at SecurityBSides Las Vegas last year and that it would be “free for anyone to download and use” (see slides 49, 50, 51 and 53 here). That particular statement makes me think of one thing in particular I learned and I will cover later in this post.
People will try to put you and your work down
This is the biggest and hardest lesson I probably learned over the past year. Most people who have used Agnitio have praised and enjoyed the application but there has of course been a small amount of people who provide nothing but negative feedback and trouble. The negative feedback and trouble has come from people in a wide range of positions in our industry including people high up in organisations such as OWASP who should be encouraging projects like this not trying to put them down. If you really want to see the personal agendas of some application security people release an open source application security tool! It showed me that we have too many people in application security who actually have no real desire to move application security forward, they gain too much from trotting out the same old lines and references and they don’t want anything to come along and disrupt that.
Let me give you some specific examples, first of all there was one particular person who appeared to want the Agnitio project banned and removed from Source Forge. He had no real reason other than I’d been slower than he wanted me to be in uploading my source code as well as the installer. This escalated to this person contacting Source Forge and reporting me for a violation of the TOS, I have to say this process was a pain in the backside to go through. If I hadn’t accidentally deleted the emails when I did a cleanup recently I would publish them to show you how it unfolded.
Another example was a blogger who praised Agnitio on his own blog and linked to an Angitio presentation on this site. I was then surprised to find this person then wrote comments which says Agnitio is a simple little database app, hard to use and saying I used the term “static analysis” (“analysing source code without executing the application” is exactly what Agnitio helps with but that was a different days discussion) just to attract people to the project and my blog. I won’t publish the private conversations I had with this person but the comments are public and on this website if you want to read them.
Comments on other blogs have also been quite funny and frustrating at the same time. Colin Watson is a fan of Agnitio and he wrote a blog post when v1.0 was released. You can see two comments on his blog which were certainly not posted to praise Agnitio. You will also see I tried to address both of the people who posted comments but unsurprisingly they never commented again or contacted me via Twitter, email etc. That is something I’d certainly say is one of my biggest frustrations so far, people give negative feedback but never elaborate on it when I’ve asked them to. If you think Agnitio is rubbish or lacking something I can’t make it better without good feedback!
Ultimately you have to be ready for people calling your baby ugly, it doesn’t matter how many hours you work on your project it will be called ugly. It hurts, it might make you angry but that’s life and I try to extract positives from these kinds of situations. I try to get more specific feedback and take the “customer is always right” stance, sometimes this works out well and other times you get nowhere.
Know your license type and TOS well!
Open source licenses are something I’d never really paid much attention to and now I’ve picked one for Agnitio I hope to avoid reading about different license types for the rest of my life. It is important to understand which license is right for your project but I struggled to find any good guides for open source licensing n00bs. I did a lot of reading and narrowed it down to a couple of different licenses but ultimately I went with one of the most popular licenses on Source Forge: GPLv3.
As I mentioned above I had problems relating to one part of the Source Forge TOS which I’m happy to admit was my own fault. I failed to properly understand my obligations for hosting an application on Source Forge. This was the first open source application I’d released and I was naïve, it isn’t as simple as throwing an installer and some source code up on to Source Forge. You have to do these things in a particular way otherwise you get reported and go through a process where you aren’t really helped but expected to just understand and fix the thing you have been reported for.
I mentioned above that I wanted to make Agnitio “free for anyone to download and use” but things aren’t as simple as that. I’m not trying to blame anyone else here, I failed to understand what license was best for Agnitio and what exactly I needed to uploaded and host on Source Forge. This caused me grief that I could have avoided if I had read and understood the TOS. Then again maybe it wouldn’t have changed anything, even when the Source Forge rep told me the specific point I’d violated I still had to push them to be more specific before I could sort the problem out.
Test, test and test again
My respect for QA testers increased a lot over the past year after I started to try and draw up test plans and execute them for each Agnitio release. I’ve always taken the approach of testing as I develop and then I try to test “everything” once I’ve finished. I’m convinced this approach has prevented a lot of bugs getting into the final release of Agnitio but that doesn’t mean I’ve caught everything. There is one bug that I introduced in v1.0 that no one found until v1.2 had been released. The bug occurred when the user went to the verify report tab and clicked verify without browsing to report first, I hadn’t included a check to see whether the path was empty before trying to open a file:
If you are using v1.0, v1.1 or v1.2 you can try that yourself if you want to crash Agnitio
I had only tested whether this part of Agnitio functioned as I expected it to or how I expected a user to use the verify report function. If you are releasing your own open source application I’d encourage you engage with any QA people you know and learn how they approach testing, I know this helped me change how I tested Agnitio and hopefully find silly issues like the one above in future.
The other testing lesson I learned the hard way was that Agnitio would not only be used on English versions of Windows. Agnitio has been downloaded by people in over 90 countries which meant some users had problems using parts of it because I never tested it on a non English version of Windows. I had regexs which were wrong and file paths in the code in English which caused users problems. One example that sticks out in my mind is a regex I had written to validate the date field on the code review tab. I wrote this based on what I could see Agnitio creating when the calendar popup was used, a date value with numbers and / in it:
This caused users of some versions of Windows problems; the calendar popup takes its date format from the Windows date/time format. So some users had a date field with Jan/Feb/Mar and so on in it and not the month number that I expected.
I love open source development and the community support
I must say that I’m delighted with how Agnitio has been received and used, constructively criticised and improved over the past year. The negative people or incidents are the ones that will always jump out in my mind first but for every one of those there have been many, many more positive interactions.
I now have even more respect for the open source community and its contributors than I had a year ago. The most encouraging sign to me has been seeing how others are willing to give up some of their free time to try and make Agnitio better. I have had individuals give lots of useful feedback, contribute their own time, perform QA testing, translate Agnitio content and even offer some the use of their employees time to help Agnitio progress. I cannot thank those people enough for their support and help but Steven van der Baan, Ryan Dewhurst, Daniel Cornell, Angel Alonso, Tiago Henriques, Jim Bird, Fabricio Braz, Colin Watson and Nabil Ouchn have been stars in contributing to and supporting Agnitio!
A challenge to other application security people
I’ve said many times before I’m not a developer, I’ve been doing “things” with code since I started writing BASIC on my Spectrum as a child and I’ve always coded to make a particular task easier. Agnitio is no different; I pulled a lot of (ugly in places!) code together to make a task easier. The point I want to make here is that whilst I don’t think you have to have been (or currently be) a professional developer to be a good application security professional I think you have to be able to write code. To be more specific if you haven’t tried to write secure code, if you haven’t had to write code that others will review, if you haven’t written code that will be used by someone other than yourself you have to ask yourself a question; why should developers listen to you and your secure development ideas? If you haven’t done any of the above you don’t know how difficult things are to implement and you don’t know what makes secure coding difficult and painful.
If you are someone who blogs, tweets, talks at conferences about application security or even have an application security job ask yourself that question and be honest with your answer. Go further and ask yourself when was the last time you sat down with a developer and tried to help them write secure code? I don’t mean giving them some generic “OWASP top ten” training I mean actually sit in front of the IDE with them and help them write secure code or help them fix a vulnerability in the code. What about helping them prototype security functionality or designing a secure application with them? How many times have you made secure coding easier for your developers? If you don’t know what makes secure coding difficult and painful you will never make a difference.
What I’m trying to say is practice what you preach! Talk is cheap, we can talk of echo chambers all day long but the problem is we fill the echo chamber with talk. Even if you go outside the echo chamber and still only talk you aren’t helping.
I wanted to finish on a positive note so I will say I can’t wait to look back over the next 12 months which I hope will be even more positive than the last 12 months. I hope even more people download and use Agnitio and provide feedback, feature suggestions and whatever else they want to contribute!