Some time ago, I was faced with a problem. I had to import a lot of data from a third party API. It sounds simple and it was, but the API was using a completely flat structure for the data. It was also using a naming convention, which was completely different from the one in my code. To top it up, names it was using were not consistent at all. I didn’t want to bring such a mess into my code, so I had to figure out a way to deal with it and transform this data into something nicer. The code I will be showing here was written in c#, but I am sure you can apply this pattern in any platform you are using.
The API was providing me with the information on a certain product. It was used to display a configurator of this product on a website, so I was also getting data about possible values and if some properties should be hidden from the user. Everything was coming as a list of properties looking similar to this class:
Let’s imagine, our product is a car and create a default configuration of this product.
As you can see, the configuration we get from the API is flat, contains all the properties in one level. Also property names are not something we could easily transform into class fields. To work with such data we have two choices. One is to use it as is - work on strings and implement many If statements checking their values. The problem with such approach is, we would need to introduce many “magic strings” (evil!). If we make a mistake, it would be very hard to find it. That’s why we shouldn’t do it. Instead, we should come up with a solution allowing us to convert this data into a well structured class like this one:
As you can see, this class contains the same information as our list of properties. The difference is, this data is well structured, works on enums, not strings and is much easier to manage in the code. Now we need to come up with a way to transform one into another. Obviously, we could implement manual mapping and convert each property one by one, but remember this is just an example. In reality, I was facing hundreds of properties. There was no way I would do that manually. What I did instead was use an Attribute class. Attribute classes are like tags you can add to other classes, methods or properties. For example, when you’re using ASP.NET MVC and want to make a method available only to authorised users, you would decorate it with an [Authorize] attribute.
My idea was to create a custom attribute and tag all configuration class properties with it, specifying name of the API property it should get data from. With that, I will be able to get all the CarConfiguration properties using reflection, read the attribute data and assign them with the values I got from the API.
This is the CarProperty class describing the attribute:
Now, I can add this attribute to the CarConfiguration class:
What’s left to do is create a mapper, which will use the CarProperty attribute to map the data.
In the first step, mapper gets all CarConfiguration (the generic param T) properties and loops through them. It reads our custom CarProperty attribute and based on its type it either calls Map() method again (if the property contains another object) or copies a value from corresponding API property - based on a PropertyType (Value, PossibleValues, IsVisible). Because the API was returning everything as strings, I had to add a conversion to desired CarConfiguration types (int, string, enum). It might sound complicated, but if you look at the code above, you’ll see it’s really simple.
Such mapper makes it very easy to add new properties to parse from the API - you just add a new field in CarConfiguration class, decorate it with CarProperty attribute and that’s it.
These days, very often we need to utilise various third party APIs. Some of these provide data in some weird formats. That doesn’t mean you need to break your coding style and adapt to these formats. It’s much better to create an adapter, an abstraction layer and hide ugly things beneath it. Using the method above is one way to do it. What are you doing to deal with bad data providers? Answer in comments below.
Michał Dymel's Picture
About Michał Dymel
Passionate software developer interested in Web Development, .NET, Angular2, Architecture and security. Currently doing remote consulting.