Tag Archives: learning analytics

My Introduction to Learning Analytics

Posted on by
1

Last Friday, I attended NERLA, the first north east regional learning analytics symposium. It was organized as a NERCOMP SIG and held in Southbridge, MA. One of the greatest benefits of SIGS is the face-to-face interaction and networking that occurs. The overarching goal of the meeting was to: ​a) build focus and collaboration around all campus stakeholders; b) Get a “jump start” regionally on this emerging topic. I’ve been hearing more and more lately about learning analytics on Twitter and elsewhere, especially in regards to blogs and other social media.

While my first exposure to analytics at work is mostly from Blackboard’s Performance Dashboard and also Safe Assign, I ascertained early on in an event held a week prior, ELI’s Learning Analytics webinar, that a vast emerging discourse of analytics is expanding beyond the confines of the LMS walled-garden. I will clean up my notes from this webinar and post a blog post in the next day or two. Important to bear in mind that learning analytics is *not* new actually. We’ve been engaged in some parts of it for a long time. But concerns about privacy need to be addressed by community in order for this work to advance.

One one the biggest takeaways is the notion of early intervention. Learning analytics help us make real-time adjustments in the classroom to enable appropriate interventions. It helps to inform where students are in their mastery of course content. Described as a “peek” under the hood, it is especially useful when students are headed down the wrong path and could greatly benefit from intervening guidance from the instructor. It has the potential to radically address key developmental moments before it’s too late. This to me has great potential value as an innovative tool in blended and online courses.

There are different approaches. At the core, they aim to build a predictive model of successful learning. One approach comes from academic analytics. It takes a longitudinal/ historic data analysis approach and incorporates large volumes of existing data from libraries and academic assessment units. Often a labor intensive venture, the intent is to build code toward data that allows for mining and extraction of potentially useful trends about how learning has occurred/or has not in the curriculum; and what can be done to make it better in the future. This approach can be lengthy and costly. Also, there are challenges of relying on student self-reporting of what they think they did vs. what they *actually* did. A graduate student speaker from Brandeis describes it as: “Don’t focus on the gathering data in the present, but use what you already have to plan for tomorrow,”

More recent approaches consider learning analytics to be a short-term and real time moment in a class with a predictive glimpse at what’s going on, and what lies ahead in an individual’s learning. What do you want the LMS to do for you as instructor? What can it do for students to make experience better? Ultimately, learning analytics provide numbers that reassure us and can help us make more informed decisions. A key challenge is to graphically represent what something is (or is not). One example from Australia shows just what a graph could look like. Another example involves using EnquiryBlogger, a WordPress plugin, where students tag their posts on the basis of originality and other criteria. Understanding data and why and how it is collected has become a life skill. Perhaps even a literacy. What are rubrics we can apply to data to inform our analysis is a central question.

The SIG leadership group also planned a very clever activity and game.

ELI Webinar: Learning Analytics

Posted on by
Reply

It was very nice to tune into ELI’s webinar on Learning Analytics. Here are some notes I took… I haven’t had a chance to wordsmith it. It’s been almost two weeks and I should just share and revisit the editing when I have time. Ok, poor form this time. But this gives me good cause to improve! 🙂

The focus was around building a predictive model. Second goal. Moodog – allow students to compare their progress with each other ((important for buy-in and justification of sharing personal info about each other.)) How original is the blog post? “original” is key. It allows for qualitative and quantitative analysis of work in high-volume classes. More real time/implement changes in real time during the semester.

The Actors > instructors, administrators (assessment folks), or even students themeselves…. Predictive functions = intervention? Continuous monitoring, keep track without constant vigilance.   different from AA > preexisting tools . strategies > immediate classroom, data visualization, user-accessible.

Resources:
Publication Simon recommended – I printed this out:
Learning Disposition and Transferable Competencies
http://projects.kmi.open.ac.uk/hyperdiscourse/docs/SBS-RDC-review.pdf

LAK 12 Conference site
http://lak12.sites.olt.ubc.ca/

Building a predictive model

A.     Mike Sharkey
###Debate is online monitoring- etc.

Lots of data is great… but foundation to extract the information you need? Have to figure that out. Turn into valuable info, then put in front of someone who can do something about it.

Capture all.  See what you can do after.. complex shifting landscape of pointing whatever data output .. need skilled people who need to know which field need what kind of info. Working well at U Saskatcheawan…
Parallel processing of data. Get all the data, then write jobs afterwards to extract the jobs you did. Ie. Did student post something right after x, did they take this class afte rr this.. that’s complex and tricky. Instead…. Throw all data in there into a Hadoop Data center and make sense of it on the backend. 40 node dual..

Hadoop, open source large data mining parallel processing dtat. HADOOP

“Can we predict if they will pass or fail this course?” 1- fail to 10-super pass flying colors. Feed this score to academic counselors.

So far, the method is:
Built different models
Strongest predictive coefficients. – identify recurring “cooefficients” and map to outcomes such as a 4 means, they are failing, advise the advisor.

Hadoop.apache.org
LAK conference lak12.sites.olt.ubc.ca

Will a predictive model be broadly applicable across all course design? Mike says, if you have a uniform curriculum, then yes, centralize curriculum yes. Haven’t tested yet, but it’s very encouraging. Humanities/STEM/ any differences.. haven’t looked yet, just treating everything uniformly.
Infomatica and Tableau… microstrategy as business intelligence reporting tool. Tableau is key on data visualization, storyboarding. Tableau helps drill down and paint the right picture. It’s a specialist tool., not an enterprise wide tool. Ben, check out Tableau. Mike is a  big fan but for small enterprise only. Would be great for us in smaller context. Mike is open for other questions.

B.      Chris Brooks
University of Saskatchewan
Looking at K12 b/c it has a lot of “untapped potential”
The grey areas between research and production
A provable science? What’s your bias first?
Capture things at a semantic level. The event at the semantic level is meaningful. So plan to and toward it. A little different take form Mike. * Make hypothesis explicit in your team so they can decide how to best collect data.  Institutional Will – show some small targeted gains in the short run and great buy in later on. Ethical to do so? Why yes, it’s about evaluation.  But.. there have been abuses. Tracking too much, you might disadvantage students in real time environment.. or instances wher data could lead a teacher deny help to those who data may show are refusing to learn or put time in.. that’s a problem, too. A lot of issues could be personal data, only show part of the picture. Dead tree learning. Social learning drinks.
Q from Malcom – ethical issue is a tough one per Chris. Students are being told about how they’re being evaluated. It doesn’t say when you email me, I’ll see a graph of everything you’re doing… that is,, in the syllabus.  Lots of data we don’t’ show instructors, b/c we know we won’t get approval. The commercial tools allows instructors to see more than the research ones that they build. ie blackboard analytics, maybe.
Another Q  from MB: for someone getting started, any advice on getting started, if you have access to a lot of data, do like Mike said, and clear things /build to data with queries. IF building a tool, then you have a lot of work. Get a course grain model going to deploy on time, so what’s the institutional will? Development cycle in the summer, then wait 8 months after active use.. let that be guide. Think in terms of EVENTS. Not number of clicks. Think what is to capture what it means to log on and od something, model best practice use of students to what’s being done during an event (ie. Completing homework in LMS)
Chris is a super good guy…

Simon B. Shum from Open University
Dispositions and Skills
Simon.buckinghamsum.net
@sbski
Exchange dataset, and analytics, to accelerate rate of growth of innovations.
Great analogy of being under knife of someone who can work well under pressure, not just takien a classs, iesurgeon

Learning dispostions – > it’s possible to model them formally and informally. Students fixed in intelligence, become brittle, that is under pressure, don’t know how to cope very well.  So he’s got an inventory of questionnaire items. (see screenshot)

7 dimensions of “Learning Power” Awesome slide > take a look at it. From web to visual analytic. 7 dim. Spider diagram. A mirror for learner to have a conversation with a trained expert about their learning. How to embed this type of model in online learning platforms? ** to further personalize it.

·      Cohort data for each dimension, give a layout of dimensions in a group, what would the intervention strategy be?  And then be able to drill down on a specific dynamic..
***** EnquiryBlogger for WordPRess
******LearningEmergence.NET  << check out this site, and also Simon’s work>>>><>
Develop categories that u assign dimensions for your online work

Discourse Learning analytics
Socio-cultural discourse analysis.
Where’s the quality learning going on??? Skim through chat logs, etc. .. different methods

KMi’s Coher – web deliberation platform.. mapping out connection and roles of people who faciliatate a dialogue, like a “Broker” between tow ideas where there’s a different understanding. Very cool!

Others can run analytaics on others to look at contributions

Thinking of students as cohorts and not just collection of individuals.
Real time analytics of students online, can it be tied to dimensions. If you never ask questions, does that put you as someone who is not the curious.. what does it mean to define analytics, to dig beneath the surface, to understand more about what’s going on.
Q from MB. = the visual analytic spider diagram is one way to represent, the enquiry blogger gives you an analytic.. in teractive blog and v isualization. To classify your blogs as collaborative working relationshiops. Allows student or educator to know to see blog posts and meaning making… now imagine, meaning making beyond a blog, but across an entire system, a learner who understands their progress in a course, but also as a learner., metacognition, metacompetencies. **** Create self directed learners who will stretch themselves because they enjoy learning. That’s what were trying to push as the benchmark of LA.
April 11/12, LA design for faculty and students.