Yak Shaving Digest 2011-08-13

This blog post is inspired by Brendan W McAdams (blog | twitter) for introducing me to the term yak shaving, Aaron Bertrand (blog | twitter) for his Connect Digests and Jennifer Lopez (not that one) (website | twitter) for expressing  her desire to see the intersect of who she follows on twitter and who follows her, which thereby made me want to do the same thing.

I was working on a C# console app the past week or so that would find the intersect of my following and followed by list on twitter. This Saturday was dedicated to getting it suitable for github, which it now is. However, I encountered several issues along the way. Because I had some blocking issues, I allowed myself to be distracted  by every minor issue  I encountered with the libraries and tools I was using for the task, for a brief period of time. The purpose of this was to report the issues, and if possible resolve them. This lead to bug reports, feature requests, and two patches. I listed them all here to add some context to all of them.

  • OAuth issues: This actually occurred earlier in the week. Twitter uses OAuth v1, which I discovered isn’t really suited for desktop apps because you have to share the secret that associates a request with your app with the world. That’s the equivalent of being required to share a pgp private key with the world. My workaround is to simple make users of my app generate their own app key (annoying, but something you’d probably be willing to do to get CLI access to twitter). If the app becomes popular I will request xAuth access for my application.
  • Sensitive info in config files and version control. This is related to the first issue. This app was not even stored in a local git repo despite me being a SCM nazi because I needed to figure out a strategy that would allow me to push an empty app.config to github. I store my consumer key/secret in the app.config, which I point out above I don’t want to share with anyone. I solved this problem by not checking the config file into version control and having a sanitized one with a different name in version control. Also, I used configSource so all my OAuth things would be in a separate config file from the main one.
  • Being anal retentive about KBCsv overloads. I am a huge fan of Kent Boogaart‘s CSV library. In my console app, I wanted to output the results as a CSV, so I could sort the data with excel.  However, along the way I got the crazy idea  to change some constructor parameters from string[] toIEnumerable<string>. I discovered this could lead to ambiguous overloads . However, I did submit a fully working patch for IList<string> and WIP for the IEnumerable<string> with some commentary and asking if it made sense to complete the work. See patch 101185 and 101186 in the KBCsv patchlist for the actual code. I’ll call this a partial victory.
  • Filing a feature request with HelperTrinity. HelperTrinity is Ken Boogaart’s general helper library, used by KBCsv. While hacking on the KBCsv source I thought, “Hey! This could use some more helper and extension methods.” So I submitted a feature request.
  • Finding  a bug in ReSharper. Digging through the KBCsv source code lead me to find an edge case that could lead to resharper reporting a false error. So I opened RSRP-274868.
  • Filed two codeplex feature requests. I went total meta and made two suggestions for codeplex itself. Both are the sort of simple things that I expect will be received by the codeples staff with either a “great idea” or “no thats so terrible for this reason”
  • MongoDB feature requests. I constantly find the BSON implementation in the MongDB C# driver useful outside of mongodb. It was useful here, but the time it saved me was eaten up filing two feature requests. I hope to implement patches for both of them.
  • Finally, one of my twitter calls broke. Twitterizer is the library I am using for twitter, and in the middle of debugging a call that always worked stopped working. Looking elsewhere on the forums shows this is not a problem unique to me. This is the event that convinced me that my time would be better spent shaving the yak. In the end, this is still an open issue, but reverting to an older version of the library in a different branch, and excessive use of git cherry-pick made it work.
  • Epilogue: The app works, but LibreOffice does not correctly import all rows in the spreadsheet. Since its all the import columns occur before the problems occur, and this can be cleaned in a text editor, I will investigate and file the bugs appropiatly with KBCsv and LibreOffice appropriately. Also, I need to get access to a machine with Excel installed. I expect Excel to be better at parsing CSV.  It seems that I was specifying space as a delimiter in the CVS import dialog. Once I unchecked that the CSV imported properly.

So in conclusion, my app works mostly, and between my bug reports and patches, I hope that the libraries and ReSharper will be improved. Also, by writing this blog post, I hope that having the big picture will be useful to myself as I polish of this twitter utility, and implement fixes for some of the bugs.