I have recently devoted a lot of time learning about data in education. It is a nascent field and ripe for new researchers and professionals that would like to contribute to the improvement of education through data analysis. There are opinions against using this approach for improving elearning, especially from the instructional designer, psychometry, psychology, education, and other communities.
But data science has proven its worth in other fields, if used correctly. Clark and Meyer, in their well-known book “Elearning and the Science of Instruction”, talk about how they came to their conclusions by running many controlled experiments. Evidently, they used data to back up the guidelines they offer throughout the book. The important thing to remember in this case is that they designed and ran experiments to get that data.
There is another approach in data for education improvement where the researcher just collects data for later analysis, without running controlled experiments. In this approach, the conclusions are tied up to the case being studied, and one can, with great difficulty, make an inference based on those results. But I believe this position is hard to sustain because there is no controlled group to test against the given hypothesis. Data mining has all these problems, but that doesn’t mean that the method is useless.
Many researchers devoted a long time and effort in finding techniques that can help work around this obstacle. It does require that one plunges into the field to learn more about them. It is expensive and time consuming to design and conduct experiments in elearning. Data mining is now offering an alternative that does require the data to be treated and handled in a systematic way to avoid bias and wrong conclusions. I do believe that the future lies ahead in this area, and those who can master it will make the most progress in elearning.