Software configuration management - Part 1

-
I hope for this to be the first in a series of articles explaining what I want in a configuration tool, existing tools, and the tool I want to build.

What is the problem?

I can't keep track of every change made to a software stack, across multiple environments, without losing my mind. Instead, I allow chaos to rule supreme, with only token efforts to keep reality in check.
Slightly more concrete: Third party vendors use a vast array of formats to store software configuration. A few examples:
  • Windows Registry(eek!)
  • JSON
  • database rows(SQL, SQLite, NoSql, NotEvenSql)
  • plain text files
  • command line arguments attached to the shortcut(why do they hate us?)
  • YAML
  • XML
The second part (across multiple environments) is also unpleasant. Things probably should be different in Dev vs Prod. But if I make a change in Dev, it probably should get tested, then implemented in Prod. Hopefully without skipping any of the other environments (beta, test, departmentA_Test, Demo, Demo_Old, Prod_DontUse_Jimbo_2017_BAK). Maybe we need to keep track of what environments we even have.

What is wrong with your vendors?!

That's not really relevant. And I'm an application guy, poorly equipped to handle the role of psychologist.

What have I tried?

The Nothing System

Pros:
  • Easy setup. It's already done.
  • Everyone knows how to use it
Cons:
  • Nothing is tracked
  • Nothing is known about environments
  • Nothing is easy to find
  • Nobody know what 'should' be in place

Office documents on a shared drive

Pros:
  • Easy
  • Simple sharing by file permissions
  • Lots of choices for formatting
Cons:
  • Lots of choices for formatting (every file is different)
  • Rapidly falls out of sync with the actual deployments
  • Requires discipline to maintain
  • No versioning, hard to tell 'why' something was done
  • Searching is poor
  • Lots of duplication

Wiki

Pros:
  • Easier to share among users
  • Simpler to enforce formatting
  • Easier to review periodically
Cons:
  • Rapidly falls out of sync with the actual deployments
  • Requires discipline to maintain
  • Nobody actually enters something useful in the commit messages

What do I hope for?

I would like to find or build a system that does for configuration what an ELK stack does for logging. The ELK stack has these components:
  • Elasticsearch - powerful search and indexing system
  • Logstash - Ingest pipeline, collects, transforms, stores into Elasticsearch
  • Kibana - Dashboard to allow you to dig into the data store to get information
Together, they collect the data from all your machines in all the formats, transform it, place it into a data store, and provide a UI to access the data.
Like the ELK stack, I want a system that:
  • Can collect data from multiple systems
  • Can collect data in multiple formats (SQL, config files, etc.)
  • Transform the data to something storable
  • Store the data in a DB of some sort
  • Provide a UI to access the data
Beyond the ELK stack, we also need version control. What was changed, when, why, who.

Comments

Post a Comment

Popular posts from this blog

Write about what you read

-
Today, just minutes ago, I stumbled across Andy's working notes . What the hell kind of organizing system is this? It's like a wiki, but only notes, with a million links, and tiny pages. I don't know what it is, or if it would work for me, but I LIKE IT. Anyway, within 10 minutes of finding this delightful little place, I found myself in a whole new  Wiki rabbit hole , or if you prefer,  XKCD: The problem with Wikipedia . The number of tabs in chrome is directly proportional to how happy I am. I am very very happy. Chrome is very very sad. Now that we've covered all that, back to Write about what you read Go read his note . Yes, that's the same link as above. It's important dammit. And way, way better than whatever I was gonna write about it. Hell, the only reason I'm writing about it is that his note  (yeah I said it's important) told me to. So, now that you've read his version  (deal with it) feel free to ignore mine. Write about what you read - No...
Read more

The Plan

-
In the last post, I mentioned that we have a plan. It's more like a guideline. Or a thing I thought, but never wrote down. Or really thought out. So, we don't really have a plan. Time to fix that. Why make a plan? To get from A to B, you need some directions. Even better, a map. When you get lost (or need some inspiration), it's good to remember: I have a plan, I know where I'm going, I have at least a vague idea how to get there. Why post it here? You, the reader, make me accountable. Even if there are no readers. This is not a new idea, it is one I stole so successfully that I can't remember where it came from. Maybe the wonderful blog folks over at Backblaze Blog . Maybe in the incredible depths of  Coding Horror . Or one of the many other places online. If any of the non-existent readers of this blog can point me in the right direction, I'd love to give credit where it is due. The Plan You know it's a real plan because because it's not just a plan, i...
Read more