Instructor Training - 2021-04-29 - BST: Day 1

Day 1

Welcome and Expectations

Why are you taking this course? What goals do you have for this workshop? (write thoughts below)

• I have heavily adapted The Carpentries’ template for a train-the-trainer course teaching open research practices. I thought it would be wise to see how the original Carpentries train-the-trainer courses work in practice.
• .We want to provide SC workshops on a regular basis at the University Library in Rotterdam.
• The carpentries course at the University of Birmingham helped me learn how to use python for my data analysis needs. Having used python for a few years now in my role I wanted to give back by training to become an instructor for the same course that I was provided in the past. +
• Want to teach carpentries courses in Warsaw, Poland and learn all about teaching them to do this as good as possible!
• I want to be able to teach something I think is highly important in all fields of research (DS or reproducibility).
• I want to give courses and workshops, but I want first to learn what are the best practices about teaching, and how I can maximise what people are going to retain.
• I want to get better at teaching software to others
• I want to organize DC/SC workshops at the University Library in Bergen, as part of “data literacy” teaching
• I want to teach carpentries courses and want to learn and improve my skills on teaching.
• I want to improve my teaching ability, not just in software but also in general. Also would like to run an SC workshop in my department, where I don’t think there has been one before
• I would like to improve by ability to teach computational skills & to run courses within my department and with groups that we are collaborating with
• Want to improve my ability to teach computing skills (including become able to teach on Carpentries workshops)
• I want to improve the teaching that I am already doing as support staff, and be able to offer carpentries-style courses at Reading in future.
• I would like to learn best-practices and methods of teaching so that I can be more confident in attempting to teach others
• I basically want to learn python through teaching other very beginners
• I would like to share what I learnt from the carpentries workshop with others.
• I’m a support staff and train lab members occasionally; I would like to learn how to teach effectively and respecting people’s limits
• I would like to learn best teaching methods
• It sounds better than the teaching certification offered at my university

Building Skill with Practice

Discussion 1: Acquisition of Skill

What makes a novice?

• Limited/ no experience
• Surface level knowledge only, relies on “recipe-like” steps / formulae without any deeper understanding
• Someone interested in … starting out to …
• Little to no experience in the subject matter
• Knows basics of the topic, but gets stuck easily/has difficulty troubleshooting
• No basic knowledge on the topics
• Someone with little experience or knowledge in a certain field
• Ready to make/ accept mistakes
• No/little previous experience
• Someone without previous experience
• Applying rules they have been taught without understanding
• Often comes with great enthusiasm to learn the skill

What makes a competent practitioner?

• Knows where to find help
• Someone with basic skills, knows problem areas, can build something
• Capable of executing basic procedures independently with acceptable outcomes, but limited ability outside of standard scenarios
• is courageous to try out new things related to the skill
• some experience, can solve some tasks on its own (but maybe no deeper understanding)
• The ability to competently perform the required task
• Can execute relevant tasks, but perhaps doesn’t understand what is going on ‘under the hood’
• He/she believes he/she is the best but d not have a deep knowledge in a given topic
• An individual who is confident with the subject
• Open to feedback, understands that they do not know everything
• Knows what to google to find the answers +
• Someone that can work by itself but still needs supervision
• Comfortable with the fundamentals and more advanced aspects of a topic
• Has strategies and is familiar with the relevant resources to solve tasks/problems
• Dunning-Kruger effect?

What makes an expert?

• An expert sees the structure of a problem due to domain-specific expertise and experience.++
• All of the above, but couples detailed knowledge with broad theoretical overview
• Knows their own limitations and understands and adapts to others’
• Deep knowledge of area; able to “improvise” using understanding of purpose and reasons behind steps, can adapt to new and changing scenarios while maintaining appropriate outcomes and adhering to overall principles / goals of task
• A deep understanding of the subject matter and knowledge of background and related subjects/information
• Domain knowledge, and experience of having previously solved related problems
• Ability to self learn.
• Knows what areas they don’t know ++
• Someone who is well informed, is confident and has experience on a certain topic. This individual is able to adjust to certain ways of teaching/explaining the field and can improvise without a struggle
• Have enough knowledge to have expectations about unknown parts
• Dominate the subject well enough – does not necessarily know everything, but knows how to troubleshoot and search for information
• Confidence
• Someone that is confident with their knowledge and is able to mentor less experienced people

Discussion 2: Mental models

• Psychology of Decision-making and Advice-taking. We conceptualise advice-taking as an information integration procedure in which incoming information is assessed for reliability using the reliability of its source as a proxy, and is then integrated via a Bayesian integration procedure with internal information (assessed for reliability using subjective confidence as a proxy). This conceptualisation misses an awful lot of the social aspects of advice-taking, and it also performs far better for discrete informational problems (how many sweets are in that jar?) than more open questions (where should I go on holiday?). The latter are often closer to the sorts of advice we often seek in everyday life.
• Functional neurosurgery - neurons communicate using electrical signals; electrical stimulation can be used to manipulate these signals; this alteration in activity results in beneficial alterations (assuming it is done correctly…). Concepts / facts are mostly electrodynamics (stimulation systems), electrophysiology (impact on excitable tissue) and pathology (link between excitable tissue activity and disease states). Relationships between these involves attempting to understand link between stimulation applied and desired outcome.
• Urban planning: it should be by definition an interdisciplinary approach, considering social, environmental, and economical aspects. For instance, considering only the social aspect, things like income distribution within a region/city, might be helpful to understand housing needs, transportation needs, etc. For this, GIS-data is very useful.
• Dental anthropology. Tooth formation and development, and how it relates to genetics, environment, disease, etc. Incorporates knowledge of oral (micro)biology to understand how these factors affect the dentition of people in the past.
• Medical meta-research: high level model is about assessing different paths to impact, and deciding that there may be high leverage in improving scientific practice. This relies on a model of science contributes to society and effectively an assumption that improving scientific practice will improve societal outcomes (which is the thing I actually want). I think it’s a fact that scientific practices could be improved (or at least something that meta-research relies on). Practically relies on taking samples of literature to infer something broader
• Bioinformatics: characteristics of technologies used + available tools to explore an hypothesis + statistical power or drawbacks of an analysis (specifically for a certain dataset) + validation of analysis
• Peter van Huisstede: Cultural history mainly based on text-analysis and use of images. Model: Images do not transport their meaning over time, so the task is to re-place them in their/a/contructed context and analyze possible meanings using textual data.
• Biological mass spectrometry: Large data sets require clear visualisation and accompanying written/spoken explanations. Knowledge of chemistry and biology are required as well as an understanding of the engineering and physics of the instruments.
• Data science - avoiding ambiguity is the crux of DS.
• Mass Spectrometry - hard technical knowledge on instrumentation, biological knowledge of the samples, chemical knowledge of the sample preparation, troubleshooting, critical data evaluation for quality control, routine data analysis, application of statistics, creative data visualization and interpretation, collaborative communication with regards to data interpretation and feedback
• For visualization, there are a number of mental models that are used: various models of Visualization Pipelines [i.e. how data gets transformed to an image that is then understood by a user, and how then modify this through interaction], taxonomies of Tasks [ie. what the user wants to achieve, such as identifying outliers], taxonomies of visualization Types/Techniques [things like a scatterplot, or a node-link diagram]; also a mental model of available tools and software libraries, and software design patterns for how to actually implement things.
• Digital humanities: we combine “classic” research strategies of humanities and computational methods
• Microbiome: bioinformatic analysis of data, building and testing models and validation as a next step/ the findings can be a reason or a result of the biology, so we need to look for all kinds of potential relationships in a community and investigate our results based on this knowledge.
• Data analysis and genetics - we go from research questions to statistical models, and need to be aware of the assumptions in our models. On the coding side of things also trying to manage large data in a robust way and automate chunks of code where possible. Knowing how to achieve the same results in different software tools, and how to explain results to others.
• Information in Telecommunications: User call and data traffic density against time and location are being taken into consideration. The distributed nature of this information availability is considered as a challenge.
• Web engineering: computer networking models, website deployment models, checklists for production-ready web apps, hard knowledge of languages/frameworks/their inter-relation (including syntax and vocabulary), some knowledge of hardware/capabilities, knowledge of probable/basic User Experience / User Interface.
• Geoscience: locations, proximity
• Atmospheric science:data science: How the data (satellite and ground based) tell a story about the atmospheric trace gas levels and therefore the impact on global climate change. This involves data analysis, visualisation and interpretations.
• Research Data Management: policies by government and funding agencies, concepts and best-practices developed by different players, infrastructure and tools available (local/national/international), domain-knowledge

Exercise 4: Formative assessments

• This sort of exercise. Low/No-stakes Quizzes. Discussion. Answering a direct question/commenting on the answer. Shared-document updating. Relating previously-introduced concepts together. Drawing concept maps.
• Multiple choice quiz
• Regular assignments based on course content (not contributing to final assessment)
• Student-led discussion groups on specific topics (small-group tutorials, individual presentations, etc.)
• asking people to predict the effect of running a command/fragment of code before demonstrating it
• Writing an essay on a topic that has been covered (and then discussing misunderstandings or something)
• Student self-assessment exercises
• Asking people to repeat the concept you were just describing back to them in different words to check understanding
• Quizzes during class
• Exercises: Fix some broken code or break the code and explain what happened. Ask learners to explain something they learned in their own words..
• Giving presentations
• Practical Quizzes
• Exercises filling in part of the task
• Asking learners to present a piece of knowledge they have learned so far
• Homeworks during school
• Polls. Quizzes.
• Quick short surveys
• let student summarise the concept
• Workshop quiz.
• Quizzes and concept map
• Having learners teach a concept to other learners
• Asking learners to self reflect on what they think they know.Self assessment.
• Let learners give feedback to each other on assignments (peer-review)
• Presentations
• Give feedback on an assignment before they hand it in for a second time
• Schematic games

Discussion 3: Handling Outcomes

mostly one of the wrong answers?

• Ask someone to explain their answer/work through the question, and highlight where errors occur (and explain why the wrong answer is wrong and the right answer is right by correcting conceptual misunderstandings)
• Go over the concept again. This time, slower and emphasizing the important parts.
• Identify where issues exist; take time to re-address topic while addressing areas of confusion; will likely be necessary to alter approach to topic / approach from different perspective; highlight why incorrect answers are incorrect to clarify thinking
• re-explain the topic to the whole class
• Try explaining in a different way (perhaps get a different instructor/helper to explain it)
• Take time to thoroughly explain the topic again, maybe adjust the next class and/or homework accordingly
• If most students pick the wrong answer, it is important to go over the concepts again, maybe using other analogies and a different approach, and ask the student why they thought the wrong answer was the right one to understand where in the explanation they got lost.
• Explain the topic further/again in a different approach and maybe ask one of the class members to explain to the class too
• Perhaps (offline) aks helpers if/how you mis-explained - they may have more context than students to see how misunderstanding arose, whilst having an external perspective
• Start with “I would also think like that at first but now lets have a look together” talk and go on one by one for each option until they agree with right answer and eliminate other options.
• For 1: After re-explaining as suggested above, be sure to include a later formative assessment to ensure that your explanation had the desired effect
• You’ll know that that answer is the main misconception among the class, so you can focus on that one - discuss how it may seem like a plausible answer (show what the error is) and then link back to what the actual approach should be
• Provide a worked example of the problem which the class do along with you.

mostly the right answer?

• Positive reinforcement; reinforce correct answers and thought process; highlight incorrect responses and ensure remaining errors are addressed
• You’ve done good teaching! Say well done and continue with new stuff, though still clarify why any incorrect answers chosen are incorrect
• complement them and move on to the next topic
• Briefly acknowledge the correctness and move on. If you’re not sure everyone fully grasps the concept, offer another formative assessment of the concept in a little while, perhaps after discussing another topic for a bit.
• Ask why the others are wrong.
• Address the wrong answers directly (will be easier when there are only a few)
• Ask if anyone struggled to answer; if no one, then perhaps the question was too easy (depends on how important/vital that piece of the training jigsaw is!)
• Make sure to still explain the topic at least briefly for the ones who got it wrong and/or accidentally got it right but couldn’t explain it
• Do not make the few who may have not gotten it right feel bad
• Mostly suggests not all - maybe get one of the participants who got it right to attempt an explanation to the group. Discuss why the other options are wrong.
• check the rest if they understood
• Ask if the question was difficult to answer, if it was too easy maybe we should rephrase the question. Also go through the wrong option explaining why they are wrong.

an even spread among options?

• See if we can use the right answers to explain things to the wrong answers.
• Ask them to discuss their answers in class and explain their answers. Point the discussion to the right direction
• Treat it similar to the more wrong answer situation.
• Get someone to explain the correct answer, explain why the others aren’t correct
• get them tell why they selected a certain answer
• Get the class to discuss. Maybe pair the ones who got incorrect answers with those who got correct answers.
• Ask one person who gave each answer to show their working/explain their answer. Ask everyone to comment on which explanation seems most coherent. Reveal the correct answer and attempt to show where conceptual misunderstandings have occurred. Consider what content you may have to shorten later in the lesson to accommodate the discussion time.
• Start again!
• Make sure you understand why people chose a certain wrong answer (maybe the question was not asked in a clear way or the answers were confusing)
• Ask again after re-explaining the concept to make sure it is understood correctly now and there is no more general confusion (e. g. next class)

Expertise & Instruction

Discussion 4: What makes an expert

1. Think of someone that you think is an expert (doesn’t matter what they’re an expert in). In your groups discuss and write at least 3 ways that make experts special or different from other people.
2. Write in the doc how this may be a problem when they are trying to teach a novice. (Think back if you’ve had any personal experience of this in a training)

Group 1 (Vanessa, Ediane, Conor, Matt, Jenny, James)

• 1. Expert has experience
• 1. Expert has a sophisticated (and hopefully accurate) conceptual mapping
• 1. Different focus as to what’s important/the end goal
• 1. Forgot what it’s like to be a beginner
• 1. Difficulty distinguishing advanced from basic concepts
• 1. Experts may extend the conceptual mapping for their domain of expertise into inappropriate domains [Engineer’s disease]

Group 2 (Ece, James, Dan, Raisa, Henrik)

• 1.Statistics professor - very good understanding of a lot of different areas within statistics
• 1.knowledge in all the details
• 1. Don’t understand how big the gap can be in basic knowledge of e.g. maths, and it’s been so long since they actually learned those things so they may not remember the process of learning
• 1. Placing things within the bigger picture can be tricky -

Group 3 (Kai, Jennifer, Bjorn, Hafiz, Emmanuelle, Erik)

• 1. The expert is able to communicate their certain topic in a non-conflicting way
• 1.The use of jargon and professional words when communicating
• 1. Familiar with areas of a topic that are not well known (fewer facts)
• 1. The professional terms used by experts may be difficult for novices to understand
• 1. Novices may find the expert intimidating
• 1. Hard to remember common misconceptions novices may have
• 1. Difficulty conveying limitations of a topic to a novice

Group 4 (Rodrigo, James, George, Peter, Phuong)

• 1. A lifetime of experience in their chosen field
• 1.Knowledge of how to find answers to unknowns
• 1.Creativity in explaining and pushing forward new ideas, ability to combine knowledge from different sources
• 2.Overcoming jargon and lack of prerequisite knowledge
• 2.gap between your knowledge and novice

Memory & Cognitive Load

Exercise 5: Test Your Working Memory

What was your score? Write your answer below.

• 7
• 4
• 5
• 6
• 7
• 6
• 10
• 9
• 6
• 8
• 9
• 4
• 6
• 6
• 6
• 6
• 3
• 6
• 4
• 8
• 5
• 5

Mean = 6.19; Standard Deviation = 1.83

Exercise 6: Improving Short-term Memory with Chunking

Write the number of words you remembered below. How does this compare with your first attempt?

• 6
• 7
• 5->6
• 6 -> 6
• 3 -> 9
• 7 -> 6 :( 💔
• 4 -> 7
• 10 -> 10
• 4 -> 6
• 8 -> 10
• 6 -> 6
• 5 -> 10
• 6 -> 6
• 9 -> 9
• 6 ⇒ 9.
• 8 -> 9

Mean = 6.25; SD = 1.88/

Unpaired T-test = t(32.0) = -.10; p = .924 😂😅 > Insert > Special Character

Paired t-test: [People may be systematically more likely to indicate with arrows where improvements occurred] True, only those with paired data included

       Paired t-test

data: y2 and y1
t = 2.8475, df = 13, p-value = 0.01372
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.3792014 2.7636557
sample estimates:
mean of the differences
               1.571429

Exercise 7: Concept Mapping

Images on google doc

Motivation and Demotivation

Authentic Tasks

• Simulating some data - can be useful with fairly simple examples (e.g. checking a statistical test). Probably not as useful as e.g. importing data and plotting it
• Writing loop function from scratch - probably not so useful as most programs have loops already (useful in the context of the specific language though as no other option)
• loading data from .csv (in Pandas)
• Gather data from external sources (Wikipedia, Rotten Tomatoes) and write/generate code to see if you would invest money in a film series (enter data, pre-process, plot, etc) – high usefulness, quick mastery; a lot of time to compose
• Cleaning data in R and plotting simple plots to check distribution of data - fairly easy to learn and very useful for most people
• Finding stuff with the unix shell (useful, takes some time to master as it is intimidating to some).
• Unix shell: put a file with specific file name/extension and content in a specific subdirectory; achieve this with multiple ⇒ fewer steps (the quicker it can be done, i.e. based on expertise, the more useful it is)
• Designing a graph to show workflow
• Learning OO python online: Every lesson was structured in a way that part of the code and concept was already done for the episode. So, joining at the beginning of the episode and not remembering part of the previous task didn’t hamper continuing with that lesson.
• Using Git/Github to do version control (hard to do at first but very generally useful)
• Writing data from r to excel or csv (very easy, very useful)

Strategies for Motivating Learners

• Strategies to Establish Value
• Connect the material to students’ interests.
  • Provide authentic, real-world tasks.
    • Simplify a real-world task that otherwise would be too complicated to do in the workshop
    • In a ‘born open’ data workshop, invite participants to bring their own code for online experiments and help them adapt it to save data to an open repository.
    • Use domain-specific examples
    • ”Provide authentic, real-world tasks.” - Poll the class to determine what topics would be most interesting or applicable e.g. Are people interested in plotting/data manipulation/etc. And then provide worked examples of these tasks.
  • Show relevance to students’ current academic lives.
    • Show how writing the code for a plot can make it easier to repeat when you change data (which can happen in e.g. a student project)
    • Or how you can use roughly the same code to plot different dataset
    • Demonstrate how the same dataset can be plotted in different ways
  • Demonstrate the relevance of higher-level skills to students’ future professional lives.
    • Invite guest speakers or simply show slides with quotes of previous participants
  • Identify and reward what you value.
  • Show your own passion and enthusiasm for the discipline.
    • Use body language and give some interesting examples to start
    • Bring your own data/material and show students how you would perform a certain, easy to grasp, task. Preferably something that would be useful to the class
    • Show your own passion and enthusiasm for the discipline: We try to use examples based on our own research.
    • ”Show your own passion and enthusiasm for the discipline”: Try to bring own material with examples. Also talk and use an “open” body language.
• Strategies to Build Positive Expectations
  • Ensure alignment of objectives, assessments, and instructional strategies.
  • Identify an appropriate level of challenge.
  • Create assignments that provide an appropriate level of challenge:
    • have a series of tasks of increasing difficulty (which may start with faded examples)
  • Provide early success opportunities.
    • Provide concrete examples early so students can see benefit of learning skills
  • Articulate your expectations.
    • Give learning goals for the session at the beginning and target the materials so it’s clear that you’re addressing those goals
  • Provide rubrics.
  • Provide targeted feedback.
  • Be fair.
  • Educate students about the ways we explain success and failure.
  • Describe effective study strategies.
    • Guide learners to resources they can use to solve problems (Cheat-sheets etc.)
• Strategies for Self-Efficacy
  • Provide students with options and the ability to make choices.
    • ask which topics they would like to explore in more depth, if any
    • ”Provide students with options and the ability to make choices” : Teachers may provide choices and/or allow the students to come up with their own related choice options.
  • Give students an opportunity to reflect.
    • “Give students an opportunity to reflect”: Allow time to think and/or exercise a problem (maybe even in silence?)

Brainstorming Motivational Strategies

• Real-world examples. How an instructor used a method that helped them in their own life/research.
• Using simple examples gradually built-up the practice of proper coding (engineering).
• Someone showed an analysis in a reproducible container which I’d never seen before and I found it motivating to have that to ‘aim for’
• Being taught through Rstudio, was able to go from basically no experience to using the basics of git, the teachers walked it through meticulously step by step and made sure everyone kept up by having a helper who would go into a ‘side room’. It worked really well and I think everyone got there in the end
• Regular (formative) practical assignments to apply theoretical material, assignments focused on applying techniques to real-world problems (usually replicating analyses from recent research with real data, etc.)
• Using substantive real-world applications as motivation for the specific content being taught
• Kept the big picture in mind and tied the different topics together.
• excitement is contagious, when the instructor is excited, learner get excited and interested in topic
• Allow some fun and jokes (not taking it too serious, still teaching a lot and with good quality)
• Rewarded creativity – for an exercise with a specific goal, but open “interpretation” or path to reach that goal, e.g. code an algorithm or a tool
• How I can use those skills to make my job (i.e. life) easier and better and quicker / reach my goals
• Back when learning loops, the instructor first showed what a loop does before going into explaining it. Once I had a view that what a loop can do, it motivated me to concentrate more on the lecture as I could already build some potential implementations in my mind.
• Clear and enthusiastic about content and delivery
• Offered additional puzzles when planned exercises were completed early
• Instructor shared that they are working on diversifying their teaching by including various avenues of teaching, i. e. lecturing, live coding, homeworks, etc. and also recorded their lessons. This vulnerability and interest in the learners created a very positive learning environment.
• Showed how concepts link up to give a broader picture of the topic as a whole and where it could/would be extended going forward
• Being enthusiastic about the application of the skill while willing to patiently teach the basics
• The excitement of the instructor. I was motivated to learn after seeing how enthused the instructor was.
• The instructor was open to using the students data during the class to show the many uses of the code/skill
• A group of instructors - more diverse, more expertise, helper available

Brainstorming Demotivational Experiences

• Instructor who clearly did not understand the examples used and could not solve issues when they occurred (and they did).
• Instructor dismissed & mocked those that did not understand / answered questions incorrectly rather than attempting to clarify / teach
• Instructor didn’t seem happy to be there having to teach novices. Also questions were dismissed by being considered “trivial” to the instructor
• Assignment (that was ultimately going to be summative) where we had three iterations to submit and get feedback before final submission - and the instructor gave no constructive feedback, didn’t even say which part was wrong.
• Completely dismissed/ignored my idea because it wasn’t the expected point of discussion
• not making clear what is the expected outcome of a specific session
• read straight from the guide and could not accommodate any questions besides those she had.
• Instructor embarrassed learners who gave unusual/incorrect answers (no safe environment)
• Teaching too many basic definitions at first, without getting to interesting real world problems. e. g. Teaching multiple weeks with data created by typing out data frames rather than loading in data.
• Simply repeating (very slowly) material which was already fully understood from pre-reading. Could be improved by implementing more active teaching methods with scope for extending beyond basic material.
• Attending a workshop pitched at R and python users, the examples in which only actually worked in python and the R workflow hadn’t been tested properly and weren’t feasible in the timeframe of the lesson

Overcoming Imposter Syndrome

• Writing multiple lines of R code that could be boiled down to a single/few lines. It comes with experience and helps you develop more robust code (I’m still embarrassed by my early scripts…) +++++
  • I wrote several thousand lines of code using for loops. When I vectorised it the code ran several hundred times faster!
  • Linked to above: I used to write several-hundred line long MATLAB scripts because I didn’t really know how function definitions worked…
  • I should also add that re-reading old scripts and being able to ‘fix’ them helps a bit with imposter syndrome
• A couple of years ago I walked out of a statistics (using R) class because I couldn’t even open a.csv file using R. I kept making silly mistakes but eventually I got it and I am pretty good at what I do now.
• Every time I find significant tweaks to my workflow (continuous integration, unit tests, package managers) I wonder how I managed before… Sometime I have to remind myself of the benefits when struggling with time to learn new things (e.g. containers for development).
• I could not come up with an algorithm asked for, but I made an outline using comments which nevertheless held a promising track towards a solution. I still work this way.
• It took me a long time to be able to actually implement version control and use it in projects. I just couldn’t really understand how to do it. Now I use it all the time and it’s just part of my workflow, not really requiring any additional effort (and in fact it saves me time and make collaboration possible)
• I always have the impression that my mistakes are silly or embarrassing so I can totally understand the feeling and make the first silly mistake so that the learners see that it’s no problem at all? .
• I feel frustrated when making mistakes, till now not know how to handle this feeling
• Didn’t know any programming as a biologist and was so afraid of Terminal or R studio screen. It was alien language. By time and failing several times, now i can teach what i was struggling.
• I learned SQL during my internship and I dropped all the tables from the database. I was very ashamed, embarrassed and thought I had destroyed all the work.
• I could share that I was not sure to even go to my first Carpentry lesson since I was “not a computer scientist” (stereotype) and felt that I would not belong. However, here I am - still not a computer scientist - but teaching a Carpentry course.
• I always feel embarrassed when I’m sharing my screen and somebody asks me how something is coded - I’m like incapable of just writing code - I need to google first or look at my own previous code :D like every single time. It’s trial and error to see if I achieve the right outcome (which is how I code but does not look beautiful when teaching live)
• I was new to SQL and was told not to create materialized views in SQL since it might crash the server. Two minutes later, I did exactly that.
• I am a bit “embarrassed” of the first project I did in Bioinformatics, but looking back there is also some pride in having achieved it and especially having improved a lot with time and practice
• I spent nearly a day debugging a problem, only to find out I had been using the wrong test data. There was nothing wrong with the code.
• Being hesitant to share code publicly (as I am scientist, not a software developer…). I was encouraged by experts in the field that it is still important in terms of reproducibility and can inspire others. The code is functional, it does not need to be elegant.
• Learning SS7 in GSM was daunting for me at the beginning. However, eventually I could master it with a lot of hardwork and practice and a lot of trial and error. Then on, I could program it to achieve things that created great services in the industry.
• Embarrassed to share my code on github for the public to see (including everyone who could have done a better job) - but transparency is a high good for me, so I keep overcoming this fear
• overwriting script file, after working whole day on it and making perfect
  • 😭

Learning About Accessibility

• Plain language and clear layoutD
• I read these through a few weeks ago while designing my own TTT course! But I really like them, both in their content and in their practice what you preach implementation.
• Always add captions and subtitles to videos. Make information clear, concise and in bullet points
• (aggressive) Time limits for anxious users
• Build for keyboard use only
• Use text and not symbols. I always found symbols intuitive and did not consider that others may not.
• Explain what will happen after completing a service to let them know what they did has worked
• Add informative alt text to images & make link text explicit re. what happens when you click (for screen readers etc.)
• keep information clear and easy to understand
• To not use overcomplicated words/language
• Figures of speech/idioms can make content hard to manage for a variety of people +
  • And can be very culture specific!
• Funny how many of those tips would just make life easier for all readers, not just those with accessibility difficulties ++++++
  • This is called the ‘cut curb principle’, after the discovery that implementing dips where the pavement slopes down into the road for wheelchair users actually afforded benefits to a wide number of other users, such as people using pushchairs.👍
  • People may also have temporary or situational accessibility needs:
  • Inclusive A Microsoft Design Toolkit.
  • 👍
• Did not consider yet how bright contrasting colours may be inappropriate
• Unfortunately, some recommendations exclude each other, you have to choose which group you want to be targeting/excluding
  • curious which ones you’re referring to?
  • E.g. no bright colors vs. accessible for reduced vision (high contrast & bright colors)
  • That makes sense! I can see that. There may be a way to find a color combo that works (or maybe just find other ways to reiterate main points)
  • They do seem to be contradictory, but one is mostly referring to text (having text legible against its background) while the other is referring to the visual syntax of elements. Agreed that sometimes they can be in tension, though, and generally high contrast helps users with low vision but can disturb users on the austistic spectrum THANKS
  • I often use https://colorbrewer2.org/ for color schemes that take vision impairment into account. Would be nice to get input if any of the schemes is less disturbing for users on the autistic spectrum.
  • Yeah I was surprised that none of the posters explicitly mentioned suitable palettes for colour-blindness
• Transfer the content by more than one channel (e.g. figures: color + text)
• Simple colours for users on the autistic spectrum
• Use simple language
• don’t use figures of speech too much

Mindset

Does Mindset matter?

• Offer various ways of interaction for learners: one-on-one, breakout rooms, anonymous feedback
• Think of it as language, and not necessarily “programming” – write a shopping list!
• Show compassion, live inclusivity by sharing that you as the teacher had similar thoughts at some point, show that teachers come from different backgrounds
• Be aware that there may be a ‘backstory’ to an individual’s mindset that we may not have the ability to unpack during a course (it may be inappropriate to tell someone to simply ‘not feel that way’)
• Ask learners about their personal motivation to join in the first place and try to remind them of that throughout the course
• Learning something for the first time is scary, especially with code! Having someone follow and guide you along the way makes a huge difference.
• Mindset can be a barrier about how people are going to receive the information, specially if they have the pre concept that they can’t do something

Helping Learners Learn From Mistakes

• They have forgotten to assign the output of an r command to a variable, meaning that it is not retained for the next step in the process. I would simplify the example and ask them to use a function they know the expected output of, and ask them how they would perform the same operation again (or a sensible simple following operation). I’d then encourage them to use that form of solution (mid-stage variable, pipe, whatever) to solve their original issue.
• They reopened the code many times. Suggest them not to reopen every time they need to run it
• In Python writing the higher/lower game using user input (input()) almost always trips people up because the function returns a string even when they typed in an integer. Tracking the stacktrace and testing for types, that play a role under the hood, make a good exercise .in “being the compiler”.
• Investigate thought process behind error (why did they do it this way?) -> aim to correct underlying model
• What do you think has happened here? What were you trying to achieve? Try to find out why they made the mistake and then help them to correct it (more empowering IMO than explaining it directly)
• I would ask if they have tried googling the error message. If time permits, get them to talk through what they’re trying to do with the command to see if looking back with fresh eyes helps them spot the mistake.
• Explain the syntax of the ls command as you are correcting the mistake
• First ask the learner if he/she really misunderstood that (ask if they see a mistake there). If it is really a fundamental misunderstanding it is possible that more learners misunderstood it. So maybe explain again
• First ask if there is any way to improve on that , see if they spot mistakes. Have you tried running it? Let’s run it now…
• I would re-explain the concept to the entire class because the error was in a fundamental concept and more people can have the same problem. Also, in a programming error I could teach how to read the error messages so the learner can identify what went wrong.
• If there is time, I would want to show them how I would go about finding the source of an error. If it involves multiple lines of code, I would tell them to run each line separately to narrow down where the error is happening. I would teach them to compare with a previous, working example to spot differences. If it is just one line of code but they cannot see the error themselves (e. g. the space as above), I would tell them where the error is but try to have them say back to me why that was an error and have them generalise it as much as they can (e. g. there needs to be a space in all cases of xyz).
• Explain what the important parts of the error message are, and how to recognise keywords that suggest the problem in future (e.g., that a TypeError suggests a variable isn’t a number/string/list/whatever you are trying to use it as)
• Ask the learner how they would approach de-bugging, help them to circle the problem. Provide enough guidance if necessary but give them the feeling they were able to solve the issue themselves.
• Forgot to go into the appropriate directory for using scp. This error made everyone learn that it is possible to do scp not being in that directory explicitly.

Provide an Example of Habits Life-Long Learners

• Always be prepared to start from scratch (example Richard Feynman), no matter how experienced you are (in a certain field).
• Focus on the journey, enjoying the process, rather than only focusing on the outcome (in other words: work for yourself, not for others)
• Sometimes look back and be proud of what you have already learned to motivate yourself
• I start seeing things as a predetermined routine until it becomes natural. The more you see the changes the more motivated I am.
• Divide work/break up a task into its smaller parts that seem more tangible
• Tell what you do to other people, the things very normal to you may impress others and motivate/remind you to see big picture
• Never assume your first successful approach is the best approach.
• Always read around and try to keep ‘borrowing’ from other disciplines and improving
• Always being willing to question what you think and accept that you might have been wrong
• Always be willing to improve/learn from others
• Accepting that there will always be something to learn, and that there will always be some level of ignorance, and this should be exciting, not overwhelming
• If stuck, seek help for a quick solution and spend some time improving on it.

Teaching Is a Skill

Giving Feedback

Content +

• Covering material
• Complete code context in view
• Too fast. “Even Excel users can understand that”
• presents all the code used for the example and builds up from scratch
• Builds complexity with specific examples
• (Trying to) give a specific example of the concept being taught
• Starting with simple examples
• fixes own mistakes
• Went back to topics previously explained

Content -

• Explains what code he is typing but not why
• Inconsistent re. Level of content & goal: introduces functions then says “for those of you who haven’t seen functions before, this is what you would expect, just trust me”; not clear what goal of presentation actually is
• Could go through material slower and providing more explanation and being more mindful of people’s background knowledge (or lack thereof)
• uses ‘just’ / ‘of course’ to make it seem like it should be obvious, better to avoid these terms
• ‘Trust me’
• lots of jargon
• Content somehow unclear (i didn’t understand what he wanted to tell us)
• Lack of structure
• He is pointing at something but I can’t see what, I doubt the audience in the class can. He should use a pointer.
• Mistake not explained

Presentation +

• Repeats what was learned/discussed before
• Fairly clear delivery
• Gives learners time to adjust/think when saying “what we are going to do now” rather than only talking content
• Tells learners not to worry if they don’t understand everything yet
• Positive presentation: live coding;
• Practical examples of what is being described
• “My bad”
• Asks for questions
• Confident
• live coding seems good and is talking while coding

Presentation -

• Uses language like “this is very simple”
• “Even Excel” :P
• Mutters under breath about students
• Talks too fast; Non-inclusive language
• Excessive use of jargon
• Unclear of what he wants to do next
• Faces the presentation often, not the students
• Too much jargon: ‘lexical binding’, ‘polymorphism’, ‘instantiating a new function object, not actually evaluating’
• Looks at phone (to check time?)
• Starts before waiting for students to sit down
• Probably better for live students but for the video he could do with zooming in a bunch, the code is too small to read!
• He obviously wants to be anywhere else right now…
• Repeated interruptions to lesson flow - interrupts at start to tell students to sit, checks phone, etc.
• Extensive use of “simple”, “just”,…
• “Trust me” (I’m the expert here)
• Too fast, jargon, dismissive/demotivating language.
• Presentation is sloppy.
• Text on screen is small - would be hard to read from the back of the room (or in a video recording…)
• Pretty fast
• Tiny text - difficult to follow. A zoomed in view while typing could make it easier to follow.
• Too many interruptions

Questions from day 1

• How does one reinforce the learning for most students without boring the faster students?
  • When I’ve got only a couple of either faster learners or already with a good enough mental model, then I try to engage them with the “slower” learners. If they are very confident on that topic, I may ask them to become helpers.
  • If there’s clearly two distinct groups it may be because the workshop hasn’t been pitched to the right audience, and separating the group into two levels may be a better option. Normally, if you’ve got the results of the survey before the workshop, then probably you will see that information before starting.
    • If you discover that during the workshop, then I think it’s better that some get “bored” than lose people forever…
• Do you find teaching material needed updating due to exclusive virtual meetings these days? And if so, what were the most significant parts?
  • The content of the material hasn’t changed that much as far as I’m aware. The delivery has changed completely though! From being able to have a full screen with RStudio, now your learners need to keep zoom, RStudio, and a chat running all in the same small laptop screen.
    • RStudio Cloud can be really useful for ensuring all users have the same environment!
  • The Carpentries have published some posts in their blog sharing experiences (not just success stories) and there are a couple of good resources (Greg Wilson’s talk about teaching online on short notice). There’s an extra module for this Instructor training that provides some guidance for online teaching.
  • Outside the carpentries, and talking here from my personal experience, the challenge has been to keep students engaged in the classroom. You may require to run even more frequent formative assessments. The problem is these extraneous loads (“distractions”) that you, as an instructor, can’t control. They don’t need to be related to the student interest, but the student situation (not good environment for studying, bad chair, small screen, etc…)
• How to reassure participants about a topic they find daunting without using the forbidden dismissive words?
  • Now you are aware of these words and the unintended meaning they can have. Then try to pay attention to when you say them. Get a colleague or a co-instructor that pays attention to what you say and take notes of that.
  • regarding how to reassure participants that find a topic daunting, then is here when you need to pay more attention to that vocabulary. If you break that topic in small chunks, provide the framework for the connections to appear, and together build the mental model of the topic, then that will help the students to learn it. If on top of that you reinforce it with an inclusive environment and growth mindset, then the students will also be more motivated.
• Teaching basics without teaching basic concepts; e.g., teaching python without necessarily teaching what an object is. How can you do this?
  • The two python lessons in the Software Carpentry curricula (programming with python and plotting and programming in python) don’t explain what an object is, though you use methods and attributes of some objects at some point (e.g., array.shape). There’s no need, for the purpose of that lesson, or for many of the things you can do with Python to really know what’s an object. Is it helpful to know about it? Yes!, and quite a lot… but how useful will it be for a researcher that wants to make a script to get something processed faster than going analysing these files one by one? Later, when that novice becomes a competent practitioner, then they could look into what objects are and improve their mental model and, certainly, their code (maybe making the code more sustainable).
• Is it helpful to teach participants practical skills without the conceptual understanding? A little knowledge can be a dangerous thing.
  • Reusing the example from the previous question, is it useful for a learner to learn to use Python without knowing that everything in Python are Objects? Or, is it helpful for learners to know how git actually stores the information to use it? I don’t think, for the purposes of our learners, these are needed. It may be dangerous, if we don’t know that numbers in programming have a limit and we can get “bitten” by that in the future (e.g., excel limitation mentioned today). Instead of teaching these details, I’d tell the learners that the goal of the workshop is not to make them experts but to create some fundations where they can build further. I’d also mention that all the tools have limitations, and gotchas, that you may affect your results. If you have doubts whether you are going to limits or what you are doing may not be appropriate anymore, then try to look for help (not necessarily only for software, but for these statistical methods, for that yoga instructor, etc.)
• How to explain concepts respecting people disabilities?
  • Does this refer to the renaming of expert’s “blind spot” with “awareness gap”? If so, by listening the community. There are many things that are named with a non-inclusive language. If you find one, or think something could be not respectful, ask the community to see other point of views.
  • If, however, the question was not about inclusive language, but inclusive space, then ask them what they need. It may require that you change a lot of your plans to accommodate a better learning of just one person, but you’ll see that the efforts you put in that will be helpful for all. For example, making the material more accessible (like discussed today in the a11y section), using a more descriptive language when explaining things (instead of clicking “here” and pointing with the mouse saying we click in the hamburger button at the top left corner), change your mouse pointer to be more visible, etc. All these changes will help all the learners with or without disabilities. The clicking examples are extremely useful when teaching online as not everyone gets the mouse movements as clearly as you see them on your screen.
• Guidelines for calculating time need of interactive exercises?
  • My way of doing this is by trying out a particular time, then measure and adapt next time. Bit by bit you start to get a “feeling” about how long it will take… That estimation will, at the start, be still wrong, so it’s better to always provide a bit more time and then if the learners finish earlier, move forward when done.
• What exactly is meant by deliberate practice? How can this be implemented?
  • From the lesson’s glossary: The act of observing performance of a task while doing it in order to improve ability.
  • That’s shown when we talked about reflective practice vs repetition. So it’s when you take the steps to practice looking for feedback. Like an archer that trains for a competition.
• How to make sure learners are not afraid of tackling material without saying ‘it’s easy’?
  • I believe that learners don’t want to learn only easy things, learning a new task has its difficulty and many times it puts us outside of our comfort zone. Presenting the material in ways that eases its comprehension is preferable than “selling it” as something easy.
• If you don’t teach concepts behind things, how do you prevent students from falling into the same traps as learning by themselves? (e.g. bad coding practices)
  • Using the example I mentioned about objects above, you can still teach good coding practices without having to learn about what’s underneath. For example, when naming variables and functions we use meaningful names, we can explain how that brings readability to the code. When teaching functions we show why they are important to avoid repetition. Though we don’t cover tests, we cover defensive programming using checks (asserts) throughout the code. From providing them these steps it gets them to a place good enough to write good and readable code. Could it be more robust? Certainly, but we are not training software developers that require years of training and practice. We are teaching researchers that can get something working and still being understandable. 👍👍👍
  • Related with this is why the Carpentries are called Carpentries. When the first of the family was created: Software carpentry, its name was chosen as carpentry is what helps to fix your broken door, engineering is what creates a big bridge.
• Should we be concerned that some of the material appears to be based on TED talks rather than pedagogy supported by robust evidence?
  • Yes, we appreciate that Ambrose et al. 2010 is summary of research that may sound like common sense and is somewhat dated, but The Carpentries strive to use evidence-based principles: see the references in Greg Wilson’s “Teaching Tech Together” (Taylor & Francis, 2019). Robustness of research is a delicate issue and we are happy to discuss particular examples within the community (e.g. on slack or on the mailings lists).
  • (The carpentries is a friendly community and based on feedback, so don’t hesitate to bring these topics to the forums or the community calls)
• For the teaching demo of an episode, can the introduction to episode be outside the timer?
  • Ideally a brief intro would fit into the slot, but feel free to add a couple of sentences for context outside the time limit (e.g. what do you expect the learners already know from previous episodes). The focus should be on live coding itself. Also try to ensure, if possible, that the notes are not visible to the audience (it is more convenient to have two monitors for this).
• format: the Etherpad was not really used enough to warrant its existence? Maybe just give straight the Google Doc link.
  • Yes, true. We only use Etherpad as backup and to relay the Docs link.
  • Etherpad are the default in many carpentry workshops, and that’s why it was on the workshop page. However, we decided to use Google doc for this type of workshop as it allows the inclusion of images. Since the workshop page had been published days before we took the decision to not use it, we decided to put the link there for those that may have bookmarked it already.
• Accessibility: is there best-practices that should be followed during teaching/live-coding? (contrast, color profile,…)
  • Yes, please ask your audience about visibility and font size at the start of the session. Please refer to the Carpentries Handbook’s accessibility section, although those suggestions are focused more on in-person workshops. Some more information can be found at the A11Y project.

Day 2

Welcome back

Carpentries logistic questions

• How to encourage people to set up software properly in advance to avoid delays during the workshops? +
  • And what resources to offer to help them do ^
  • There are detailed setup instructions available as part of the materials for different operating systems (Windows, MacOS, Linux) and we ask participants to install everything in advance in a pre-workshop email; sometimes we run pre workshop tech sessions on the day before (for an hour or so as drop-in)
• How extensively do you prepare the lessons, particularly if you teach them for the first time? +++
  • It is good to go through the lesson in its entirety at least once. It may be worth joining a demo session an test-teach an episode. We need to make sure that we know the materials well, sp that the learners can have confidence in the instruction. -
• Central vs. self-organized workshops. I understand that centrally organized workshops follow the established lessons but the contents can be more freely combined/adjusted (incl. “concept”-material) in self-organized workshops? How about combinating material for SC/DC/LC?
  • In general there is considerable freedom in how the materials can be combined according to the learners needs, but to be a Carpentries workshop it needs to follow approved curriculum. And there are some definitions on what it means for a workshop to be e.g. Software Carpentry: it need UNIX shell and version control along with a programming language (R or Python)
• What are all the spaces ((internal) websites, mailing lists etc.) that carpentries instructors use to communicate? What do I need to monitor to be up-to-date?
  • In general there is the ‘discuss’ mailing list, but also slack, topic box, github issues and other channels (we will come back to this in the afternoon)
• There are several quite interesting other lessons (like FAIR Data&Software) but on status “concept”. How long will it take to be on status “alpha” (to give them)?
  • This depends mainly on the rate of contributions by the community, but it is difficult to tell. It may be worth organising a hackathon type event to get some momentum going.
• Do you use virtual machines to be sure everyone has a working system with the same version?
  • People have tried using virtual machine in the past but to prove to be a extra cognitive load and it may run slowly to require some non-trivial configuration steps. We also didn’t want to maintain instruction for different virtual machine software (Similar experience for containers). On the other hand RCloud setup seems to work well.
• How to co-teaching with others in one class?
  • It is common to split the workshop into lessons and for instructors to be main instructor for different lessons.
• It would be very useful to have a short “looking over your shoulder” demonstration of teaching a lesson - what are your notes, how are your windows organized, what are your tricks to remember things, how do you decide what lesson material you literally repeat from the lesson/highlight/skip etc.
• Can you do the R workshop in Rstudio or is generally done just from R? And are you supposed to do it directly in the console? Seems strange to me not work from a script +
  • We start in the console to reduce cognitive load but in general then move on to use RStudio, because it is the de-facto standard environment in practice.
• Online-teaching and multitasking, should this explicitly be mentioned? (this course is a good example, things happen on Zoom & chat & Google Docs at the same time. Even if you closed your email inbox, which you likely did not even do)
  • Online there may be more things to keep track of and we try to split the tasks among instructors and helpers (e.g. monitoring chat and answering questions there while instructor is moving through the materials) [definitely hide all other apps and turn notifications off ]
• Live coding: is copy & paste appropriate in some cases or is always to be avoided?
  • It may be justified sometimes, but use it sparingly. We don’t normally copy code off a website, but we may copy paste a function we have written to define a slightly different function faster (as it illustrated a common practice).
• How to determine the optimum time that a workshop should last for?
  • Most lessons have time estimates associated with them but in general those tend to be a bit on the optimistic side. Leave some slack and plan what material you may drop. There is still a discussion going on about the best format: for instance for online workshops people tend to prefer different setup from what is our typical setup for in-person workshops (e.g. running a workshop as four half days may be a good idea).

Live Coding is a Skill

Exercise 1 - Up and down

Advantages

• Students get to see actual process of carrying out task in a real environment; no ‘hidden steps’ when they go to attempt exercises themselves
• Students can ask for assistance as the instructor is coding, should they be stuck..
• Learners can see that everybody makes mistakes, even the instructor. Also, they can see how to maneuver your way out of the error when that happens. +
• Instructor gets to find out what common mistakes are, means they can improve their material/delivery for next time.
• Learners see that mistakes (even typos) are pretty normal +
• Gives the opportunity of instant feedback from an instructor point of view.
• Learners will find errors that may otherwise be prohibitive for them to solve alone by themselves / in the same way the instructor learns about novel challenges that learners may have and can directly address them
• You get feedback on the nuts and bolts; bottom-up learning; easier for instructor to take steps back and discuss principles.
• Instructors and learners make mistakes together. .
• Learner can see the error messages and also how the instructor choose options and so on
• Learner gets a safe space to try running commands in a structured way with known expected outcomes and help available if needed.
• Instant feedback from students who are struggling with a particular topic
• Prevents the teacher from progressing too fast.
• Possibility for intervention & ask questions about intermediate steps that may easily be missed otherwise; more inviting to ask questions that seem “small”
• More engaging for learner - watching slides can make you zone out, actively participating is better
• Show the learners how we really do the coding in practice

Challenges

• Requires more concentration to correctly type code (potentially on unfamiliar keyboard) than to show a slide it is already written on
• Many live participants, many potential live errors
• Unfamiliar learning environment for many - most students likely to be more used to more passive lecture formats, need to ensure engage properly with more active learning & participation
• Having to stop/start the lesson to help students who are struggling which could slow the lesson down for others. Finding a balance between helping and continuing with the lesson.
• Unconfident instructors might show get confused (thought this can be good for demonstrating fallibility)
• Keeping the spontaneity and ‘live’ feeling after having given the workshop a few times.
• Going too fast..
• .It can cause the learners to lose track if the instructor is too fast.
• Instructor may be stressed by the focus on themselves and thus teach less effectively (wanting to get out of the situation)
• The instructor can be distracted sometimes because technology has a way of not working as planned.
• It can be hard for learners to keep an overview over all the material covered so far, if only a small slice of code is visible at a given time
• Some learners may feel overwhelmed by trying to watch, listen, and code at the same time ++++++
• Keep the herd together; show-off and disappear in hyperspace (leaving out these small but important steps) leaving learners on an abandoned planet.
• It could be challenging to catch up if the learner miss a part+
• More difficult to keep up, as not everyone will have the same pace; can quickly feel overwhelming+
• Instructor might lose the thread
• Temptation for instructors to go off topic & explain important but irrelevant asides - e.g. as discussed previously, temptation to explain objects & methods etc. instead of just moving on; could become overwhelming and unfocused for novices+
• The fear of forgetting some code syntax for new instructors. ++
• Challenge for experienced instructors to follow best-practice and not their usual routines (e.g. tabbing) ++
• Requires minimum two screens or excellent memory
  • Or a big enough screen for multiple windows at good resolution.
• In R, at least, encourages use of the ephemeral console rather than saving code in a file for later reference (History is inadequate and not easily transportable)
  • Could code in a script and run line-by-line and encourage them to imitate
    • Script in a project in an RStudio.cloud instance :)
      • 💜
    • RStudio is introduced rather late in the basic R lesson
• Is it actually useful to watch coding as opposed to diving in and doing it?

Exercise 2 - Compare and Contrast (10 min)

poor live coding

Content +

• Content is ok; it is all in the lousy presentation
• Explains that variable name can be anything ++
• Explained why new lines start with ‘>’ during the loop
• Repeats example three times with only small iterations to showcase what one variable change is doing (or rather not doing)
• it explains how to identify when the command was finished or not
• Uses typo to repeat loop principle

Content -

• Could use error as an opportunity to teach troubleshooting+++
• Not full screen window
• Jumped straight into his coding and no proper initial explanation

Presentation +

• good pace
• Speech is clear, and no hesitation or filler words
• Says what he is going to do before doing it
• Happy voice

Presentation -

• Doesn’t explain what he is doing as he types+++++
• Doesn’t highlight the section of code he is talking about during explanation
• Going fast and not waiting for any feedback.
• Too many other things on the screen
• And terminal is configured with a long, complicated prompt (rather than just $)
• He isn’t checking if the students are following successfully.
• Too small text size to follow.
• Phone is distracting, does not apologize
• And pop-ups
• doesn’t explain as he writes the code
• Leaves it up to learners to understand most of the code themselves
• Distractions for the learner in the form of pop-ups and the instructors phone dinging
• It did kinda slow him down 😂
• Doesn’t explain errors
• No interaction with learners or screen, way to fast, reading the script aloud typing along
• Ignores red post-it on student’s computer +

good live coding

Content +

• Funny names like unicorn+
• Made a typo and explained it+++
• And uses this as an opportunity to explain not just the typo and erorr message, but also command history, and alternative single-line syntax for loop +
• Completely explains the steps of the loop
• Explains exactly what the error is and what it means. Also how it can be fixed.
• Explain when we “need more” on the command line. Explains how each variable works inside the loop+
• I actually learned how to do it this time
• Explain the whole process in detail

Content -

• Runs the full loop instead of explaining before running the command.
• variable name could have a more explicit meaning to improve readability of the code

Presentation +

• Moderates what he is doing
• Text size is readable.+
• Good contrast of text on background
• Highlights what he is talking about to help learners follow
• He is standing +
• Paid attention to the red sticker! +
• Leaves laptop to show on the whiteboard
• Read code out loud as he types it +++
• Looking at audience
• Explains what he wants to do first, then executes it++++
• Used sticky notes to indicate participant feedback

Presentation -

• A nicer terminal would help (with syntax colouring but not the just-path-colouring from the previous one)
• Could have used full screen
• Could have used the error as an example of how to go about finding what the error is based on the error message rather than revealing right away that it was a typo (seemed almost a staged typo :D ) ++
• Is it good practice to stage typos for explanation purposes - is it better to pretend its an accident or would students find that patronising?
• Could use laser pointer/mouse to avoid walking across screen
• He is using his hand to point instead of a pointer.

Preparing to teach

Exercise 4 - Evaluate Learning Objectives

1. Identify the learning objective verb. How specifically does this verb describe the desired learner outcome?
2. Where does this verb fit on Bloom’s taxonomy? Do you think this is an appropriate level for your learners?
3. In your opinion, does the lesson do an effective job of meeting the stated objective?

Group 1 - Lesson: Working With files and Directories (The Unix Shell, Software Carpentries)

1. Objective 1: “Create a directory hierarchy that matches a given diagram” – The learner outcome seems to be more in line with being able to apply and reproduce a folder structure.
2. The verb is “create”, but it seems to ask something that is more on the “apply” level - “interpret” the diagram and “implement” it
3. Maybe not?

Group 2 - Lesson: For loops in python; objective: Correctly write for loops to repeat simple calculations.

1. ‘Write’ (correctly) - specific & testable (essentially writing correct syntax)
2. ‘Remember’ (syntax), also ‘apply’ if applying to new tasks; appropriate for intro task (largely question of basic syntax)
3. Yes - describes syntax & shows multiple examples with different tasks, exercises included with application to new calculations, etc.; next step: consider contrast with other “loops”, particularly vs. “while” for looping (analyse / evaluate)

Group 3 - Lesson: Software Carpentry - Programming with R

– Objective: “Use a for loop to process multiple files”

1. Use is the verb. It identifies the outcome as a practical one in which the learner has acquired a new skill.
2. Apply level. The skill is something learners should acquire by the end of the workshop. The learners don’t need to analyse, evaluate, or create for loops because these concepts involve a deep and nuanced understanding of many other wider programming concepts which cannot be acquired in the given time.
3. There’s a jump in the material which might disrupt users’ ability to follow. The lesson tries to ensure learners understand the concept before they use it.

Group 4 - Lesson: Exploring data frames - R for repro scientific analysis [remove rows with NA values]

1. Remove
2. Probably in ‘create’ in the taxonomy as make a new object, but this ‘feels’ too high in the hierarchy. Can also consider it as ‘remember’ as it’s a relatively basic pre-exisitng function
3. You learn how to remove rows with NA, so i guess it does…

Group 5 - Lesson: Data Types and Type Conversion

1. Define - defining the types of data
2. Remember and understand
3. Yes. To execute the understood code in their work. They can try it on their own projects or come up with a similar example to implement.

Group 6 (James, Erik, Hafiz) - Lesson: Creating functions: Define a function that takes parameters.

1. Define - A function in python
2. Remember - Syntax of a function
3. Yes - The lesson defines each statement in the function

Group 7 (Peter, George, Bjorn) - Lesson: R as a calculator in RStudio

1. Understand: Operate within the selected framework (RStudio)
2. Apply: Using operators and variables and see the outcomes of those in their particular spots of the framework

Exercise 5 - Where are your checkpoints?

Group 1:

• “Write a query to select all values for specific fields from a single table.” - assess by asking students to do the task “Write a query that selects only the name column from the Site table.”, and use a sticky note to signal success. Learners should be able to do this after the point in the lesson where SELECT queries are introduced.
• “Create a directory hierarchy that matches a given diagram.” - Sticky note to signal success/problems after returning expected (or not) output with “ls -F”
• “Configure Git to ignore specific files.” - give them various preset combinations of files and ask them to create a .gitignore that achieves the desired ignore combination. Signal success/failure with sticky note or online equiv.

Group 2:

• “Describe what a data frame is.” - Challenge: What is class of the object surveys? How many rows and columns are in this object? This could be asked by MCQ or other assessment. (learners need to be able to apply str()) - may not necessarily be able to completely describe all aspects of a data frame as that comes with experience
• “Write and build queries”: The objective should be met at the start by writing a simple query to start with. The objective will be met for all learners when all of them are able to create a first query without errors and explain the components of the query. Assessment: MCQ asking about which keyword to used and how to select all columns, for example
• Correctly write for loops to repeat simple calculations: basic syntax learned early on (check in: e.g. post-its, any issues with basic running); extension to further applications at roughly 2/3s (check in: MCQ, e.g. ‘how many times will loop body execute’, ‘what is value at end of loop’); application to repeating new calculations at end (final exercises: implement calculation to produce specific result with faded examples)

Group 3:

• “Correctly write for loops to repeat simple calculations.” - formative assessment with sticky notes
• “Trace changes to a loop variable as the loop runs”. This should be achieved where the instructor tells learners that tracing variables as the loop runs is worth doing. No exercise or formative assessment is used to verify that they can do this, and no automated help is offered (print statements, debugger). It is currently not clear whether learners can do this. A formative assessment which might help is to take an example and go around the room with learners taking one iteration of the loop each and explaining what happens with their loop variable.

Group 4:

• Remove NA rows. Should be able to do it by this point na.omit(cats) . Check by using sticky notes to see if everyone has been able to omit the relevant row?
• Challenge 1 will let the learners test if they understood or missed some points.
• Defining a function: Should be able to do this by the first or second function definition. Should be able to tell if this is successful by giving an input and seeing what the output is as part of formative assessment. Then followed by an exercise to test this further.
• During Challenges, we will use sticky notes to see what stage are the learners
• “Describe what a DataFrame is in Python.” It is probably difficult to track if the learners achieved that because many outputs of commands are not given in the class

Group 5:

• “Define data type” - Analyze - “Perform basic mathematical operations and summary statistics on data in a Pandas DataFrame.”
• “Create simple plots.” - using sticky note (green is done/red is having issues)
• Met objectives: all students have green sticky notes to show they have achieved the objective. Should half the students have red sticky notes then the objective was not successfully met.

Group 6 (James, Erik, Hafiz):

• “Define a function that takes parameters.” - This should be understood before moving on to the “variable scope” section. Could use faded examples and ask the room to fill in the gaps in the function or ask them to identify the output of simple functions to ensure understanding regarding the syntax of functions.
• “Explain what for loops are normally used for.” This lies in the explanation at the start and not necessarily the coding. The exercise at the end gives plenty of opportunity for the learners to practice the use of loops. The exercise also mixes the loops with material from previous episodes which helps learners to see the usefulness of for-loops clearer.
• Create a local Git repository: Objective is to create a directory and initialize git. Doing the ls -a command is the checkpoint where the object is achieved. The learners also get the same perspective and are most likely going to be clear.

Group 7 (Peter, George, Bjorn):

• Learn how to name objects (in R): Ask what a good name for the next object would be (possibility for MCQ or type in chat).
• Peter: remember: basic syntax of working with lists in Python; understand: lists under the hood; apply: work with lists. Roundup: analyze: So lists are workhorses especially good for …

• George: “Understand that files and folders listed in .gitignore file will be ignored by git software” » ask audience expectations of output of git status after creating gitignore (in comparison to seen output before creation of gitignore)

Workshop Introductions

Exercise - Your Academic Past

LIKED the course

• How did those courses start?
  • Instructor seemed to take the time for introductions and not focus on content yet (sat on table, chatted with us for some minutes, had presentation about outline for the course)
  • Clear explanation of goals & illustration of “end-point” - i.e. what we are building towards over the course of the lesson to inform motivation
  • Got us each to fork a small repo that contained all the materials needed, that we would be working on and building up ourselves throughout the session - helped me feel invested from the start
  • Instructor briefly went through the prerequisites and then explains the objective and a brief story of the journey that we would be experiencing
  • By basically showing what could go wrong given the topic and then building up practices to avoid all traps and pitfalls
  • Best start I ever had: Everybody was asked to name their most hated vegetable
  • With an icebreaker which helped to relax and get motivated
• Were you confident in the instructors abilities?
  • Yes
  • Yes
  • Yes - demonstrated clear understanding of topic & ability to apply topics to problems (i.e. mastery of theory & application and ability to deal with questions well)
  • Yes
• Did you feel like they were enthusiastic about the course and invested in you?
  • YES.
  • yes
  • Yes; clear desire to convey information and make students appreciate why they enjoy the topic; felt “sharing” their enthusiasm
• Was it clear what you were going to be learning?
  • Yes, instructors explained well at the beginning but also during the each sub-section what we would learn
  • Yes, right from the start
  • Yes, in written format, even ahead of the start of the course
  • Yes - clearly outlined goals, outline given in advance, clear plan for each session with goals and follow-up exercises
  • Yes, and there were people on hand to help you when needed, which was made very clear at the beginning
• Were you excited about the material?
  • Some parts were very interesting and exercises made it fun to learn and easy to remember
  • Yes, very close to our research then
  • No, presentation about presentations. A presenter that didn’t use a single word in presentations. I don’t do that but have been affected by it.
  • Good presentation made it engaging; created enthusiasm re. Applying to own work

DISLIKED (the course)

• How did those courses start?
  • The instructors were talking amongst themselves and did not introduce themselves
  • Instructor didn’t care if everybody was there, there was no clear start.
  • Poorly motivated, poor learning goals; unsure what would be taught or specifically why
  • Stepping through code rightaway
  • Were you confident in the instructors abilities?
  • No. Instructure started writing on the blackboard straight from notes. No reflections, no ability to answer questions, etc.
  • Not really, they were more concentrated on the technical things and their notes
  • No; demonstrable lack of mastery of topic
  • No.
  • Did you feel like they were enthusiastic about the course and invested in you?
  • No, the instrudctor was kind of indifferent (didn’t care about anything)
  • no, They never asked for feedback. Nor did they check if we were followingg
  • No
  • They clearly wanted to be somewhere else
• Was it clear what you were going to be learning?
  • No, I thought there would be stuff relevant to R users (as that was how the workshop was advertised) but in the end it was much more focussed on python, which I don’t use
  • No. The content was not inline with what I thought the course was about
  • Not really
• Were you excited about the material?
  • Yes, but it was mainly the presentation and lack of context

Exercise - What’s in an Introduction?

What do you hope to accomplish in a workshop introduction?

• Goals and context: What’s at the end of the rainbow? Clear steps to get there.🌈
• Inspire confidence that the students are leaving the workshop feeling like they’ve learned what they wanted to learn.
• Motivation for the course,
• Get people to look forward to what is going to follow and excited about what they can achieve. Don’t put them off
• Understand participants expectations and ability/experience
• Get people to know each other a bit
• Background of the instructor to give the students some confidence
• Everyone feels welcome, everyone ready to begin, clear starting point; motivation for lesson & why important / useful; clear learning goals; puts students “at ease”, not rushed/urgent
• Give clear structure of lesson, including breaks.

What information do you need to include in an introduction to accomplish these goals?

• Introduction to the instructors: names and brief backgrounds
• Topics to be covered, and assumed prerequisite knowledge
  • That sounds a little stressful - if I don’t know that at this point what am I gonna do about it?
    • For carpentries workshops the prerequisites are pretty minimal, so this is probably unlikely to be a problem
• A full overview of course materials which will be covered over the workshop.
• Timings - when breaks will be
• Whether laptops should already be set up etc.
• what will be accomplished by the end
• Explain end goal
• Logistics like where toilets are, escape routes in case of fire alarm
• good/structured schedule
• Overview of goal for the session. Do the learners have the correct expectations?.
• reference pre-workshop survey to address what you know about learners and their goals
• Who to seek help from if required.

• Checking everyone is present before starting

• Explaining use of post-its
• WiFI network name and password
• Ask people to mute phones
• Avoid assumptions by learning what is the expectation of each learner form the lecture

Putting It Together

Exercise - Summary Overview

CONCEPTS

• Plausible diagnostic foils in MCQ formative assessment ++++
• Dismissive words +++++++
• Memory & types of cognitive load (extraneous, intrinsic, germane)
• Factors affecting motivation (task value, task difficulty, environment, structure, accessibility) +
• How many different kinds of accessibility there are to consider in presentation format. ++
• The power of language and specific words like “just, once, any question?” ++
  • (It may well not be words, but attitude, and that paying attention to words affects attitude)
• How to build skills
• Goal oriented with clear steps; check these with formative assessments and feedback +++
• How to motivate learners++
• Blooms taxonomy and how to incorporate it into teaching +++
• Soliciting feedback ++
• Forms of formative assessment
  • Frequency of formative assessment
• Inclusivity during presentation
• Effect of learner’s mindset.
• How color choice can be more inclusive +
• The introduction of course +
• Formative vs. summative assessments
• Cognitive and Mental Models +
• Designing content based on specific learning goals & concept maps
• “Problems” of being an expert ++
• How and where to put assessments to be sure everyone is on board, also assessment types
• Expertise in X !== Expertise in Teaching X
• Do reflective practice, not repetition
• Dreyfus and Dreyfus model for skill acquisition
• Stereotype threat
• Being aware of diversity of the learners
• So many small “common sense” tips that will add up to make a big difference
• Concept maps for lesson design

SKILLS

• Live coding +++++++++++
• Doing an effective introduction +++
• Debugging guidance without taking over ++++
• Addressing accessibility +++
• Different ways of giving feedback (one up one down, 2x2) ++++++
• Fading examples; correct faulty code; serious preparation; do not use words like easy, just; use illustrations to explain difficult concepts
• Several tricks & tools/features useful in digital teaching +
• Using mistakes as teaching opportunities. +++
• Useful formative assessment tools
• Different kinds of tools for online teaching: etherpad, Google Doc, ideaboardz, mentimeter, whimsical, zoom annotate, OBS, socrative, pingo, carnac, rubberduckdebugging +++
• Importance of engaging and enthusiastic presentation

TEACHING ONLINE

HIGHLIGHT

• Being able to participate with my own computer set up - instead of eg having to bring a laptop to a session. +
• Saves the trouble of having to commute for a while, to and fro.
• No travelling (and having my own coffee) +
• Meeting people from more diverse backgrounds +
• Accommodating communities from relatively remote zones. +
• Accommodate more participants and helpers +
• Easier to participate spontaneously +
• Can participate when sick (and not transmit to the whole workshop) +
• Hour lunch break is good.+
• Typing in chat/document instead of having to speak reduces barrier to participation for shy people ++

LOW POINT

• Potential technology failures .(webcams, microphones, internet connections…) +++
• Having to restart occasionally when transferring into/out of breakout rooms.
• Not being able to see or hear what the students can see/hear+
• Sometimes not being able to see students’ faces after enabling screen sharing (depending on system used)
• Lack of informal interactions with other learners
• No cookies (+) and coffee?!
• Ordinarily, watching a presentation online means we can’t see the presenter - or having the presenter visible in a second window reduces the size of the presentation (your head-in-a-circle approach was really good! Exception to this low point!).
• May lose some participants more easily (too shy to ask for help, distractions from the household etc.)
• Breakout room discussions were quite stilted, especially afternoon.
• Constantly multitasking
• Afternoon sessions were less motivating than mornings - perhaps due to presenter audio quality + enthusiasm.
• Zoom fatigue ++