The road so far
This is the second post of our blog series regarding data analysis. This article is focused on the first step of the data analysis, that is goal setting. In our next posts we will follow all the process required to correctly analise data, while our previous post has introduced our path just started within the set of procedures to be applied to a dataset with the goal of extracting meaningful information.
Figure 1: Data Analysis main steps.
So far, so good
So, you have a question. It can be a business or a technical problem, does not matter as long as you can identify it. Actually, here is the 80% of the problem: you need to state the problem very clearly and without ambiguities, if you really want to solve it.
A list of questions can help you in finding the way:
- What do I want to accomplish?
- For what reasons?
Up to here, we are in a “qualitative area”. These questions are fundamental and are the basis for the building, but you need to add the roof: that is the “quantitative area” questions:
- Can I state it as a numerical question?
- How can I calculate it? Do we already have a function, a KPI, a variable addressing the numerical question? If we do not have it, can we design it? Also, you can change it while going on. The point is to try to define it (at some time, you will also succeed ?!).
- What is the accuracy and/or the tolerance I envisage for that numerical result? Can I use standard statistics such as relative errors, absolute errors, RMSE, NMAE… Also, you could be the first to try to assess the accuracy, or that, at first, you do not have a clear idea on how to assess it. That’s not the point, the focus is to keep it in mind (and have it ready) by the end of the activity: if your result turns out to be 5 with an error of 200%, you should not feel very proud of your approach…
- Once I have the final result, how can I validate it? Do I have some hints looking at literature? Do we have in our team similar activities so to compare results?
- Do I have the technical tools to calculate it?
- Do I have the time to meet expectations? Deadlines are always coming!
- Can I ask for a double-check to someone I trust? (please, do not re-invent the wheel: they already did it some time ago!). If possible, ask someone you trust to double-check even your initial assumptions, not to fail just at the beginning.
Let’s give some examples on solar activities
- Basic example: calculate your plant performance using the PR
- “Advanced” example: power forecast of your plant
Figure 2: example 1: do I improve my plant performance?
Figure 3: example 1: do I forecast my plant production?
If you want me to use some fancy words, I can say that we are essentially rephrasing the SMART goal setting approach (specific, measurable, agreed, realistic, time-phased) in the data science context.
Now, you are ready to go: what is your question…? Do you already have a KPI answering the question…? How do you calculate it?
i-EM at Work
i-EM provides advanced big data analytics powered solutions for intelligent energy management, enabling the optimization of energy decision making for Solar, Wind and Hydro plants and for e–Mobility, Smart and Micro grid. All company services embrace the value of the data analysis process, from analytics to data visualization to better give all the elements and forecasts information useful to make the optimal decision.
Get the best from your data.
Below, six innovative data analytics strategy for each energy asset:
h-EM provides data analysis for hydro.
Create more value from data, through big data analytics, Machine Learning and Artificial Intelligence. From data quality check to efficiency analysis to improve the service output in Hydro plants.
s-EM provides data analysis for solar.
Get the true value of data and use them properly. The suite of i-EM solar solution increase yield and decrease operational costs, maximising performances with different services: power forecast, predictive maintenance, smart monitoring, sensor check, satellite-based plant construction monitoring and UAV data management.
w-EM provides data analysis for wind.
Keep optimal performance, creating more value from data exploiting big data analytics, machine learning and artificial intelligence technologies. Wind turbines continuous monitoring and real-time failure prediction.
g-EM provides data analysis for grid management.
The innovative analytics decision support system for improve the management of the grid is empowered by satellite communication and Earth observation.
m-EM provides data analysis for microgrid management.
Advanced analytics works with the strategic complicity of the exploitation EO satellite data and SatCom links to efficiently manage urban and rural microgrids.
eV-EM provides data analysis for e-Mobility.
i-EM data analysis ensures benefits to all the electric mobility actors. Advanced analytics process works to get smart balancing, energy charging & routing and charging points planner
GET IN TOUCH NOW
If you would like to learn more about us, to ask for any information or get further details about our work in i-EM, write to us or come meet us.
See other parts in the series
Our Data Analysis blog series continues: follow all the steps through the set of procedures required to correctly analyze data and get meaningful information from them.
In the goal setting step you need to state the problem very clearly and without ambiguity. A list of questions can help you in finding the way, divided in three areas: “qualitative”, “quantitative” and “relationship”.
In the data acquisition step you need to focus on your goal and follow some tips to set your work. They are: 1) estimate the size of raw data; 2) create a skimmed dataset where to perform preliminary tests; 3) get the appropriate tool for your goal; 4) check carefully the time-zones; 5) check the files format; 6) check the language. The right questions and some simple advices can reduce timewasting.
In the data acquisition step you need to focus again on the kind of information you want to look at, carefully check the statistics and then set the right plots to comprehend the dataset. At this point, you will be able to try to understand what your data have to say. In fact, this step is often called Data understanding.
In the data pre-processing step you need to make sure that your data can be used effectively from the model you chose to use, doing specific operations, which can be divided in four areas: data cleaning, data tagging, data munging (wrangling) and data transformation. Actually these operations are connected between themselves, and sometimes you do not really need to do all of them (also, not in that specific order): it is more like a continuously refining procedure.
In the data processing step you need to decide the accuracy and the information you are interested on, choosing the models that can be successfully applied to the situation, thinking about how to validate your results and taking in account you expertise in the results interpretation. In fact, this step is also commonly referred to as data modelling.
In the validation (interpretation) step you need to keep in mind several questions aimed to understand if your work is reasonable and if the target performance is reached. So you will need a preliminary idea on how to validate the model, using a combination of three areas of validation: a “statistical area”, a “systematic area” and a “common sense” area. At the end of this step you will have your results and some kind of uncertainty associated.
In the fine tuning step you would look at results and, if you are not satisfied, you go back to previous step.. At this point you are aware of the lessons learned and you can do it better, following some tips: keep documentation, named the documents in an easy-to-remember way, try to think if there is an underlying problem you did not consider before and think about minor improvement or modifications which can result enough.
In the delivery step you need to make all of the final activities needed to provide some outputs to the customer with a fixed frequency, before sharing the delivery with the customer. This procedure depends on the specifics feature of the activity and it includes also a monitoring phase and a business data visualization work.
The reporting step is where you need to share your results and lessons learned with other people, with two goals: the first is to make clear results to yourself, while the second is to share them and raise external feedback.
And to have an in-depth look at this issue,
download our Data Analysis white paper
Data Analysis: "The Treasure Hunt" Uncover hidden gems in your data
Who should read:
Solar Asset Managers and O&M Contractors
For the curious costumer
At i-EM S.r.l. we have what you need (we hope!)… Try us!
Reading Time: 15 minutes Satellites: what they are and how we use them
Reading Time: 3 minutes A brief recap of satellites world
Reading Time: 3 minutes Floating Photovoltaic applications state of the art
Get notified of new articles