Programming and maths resources

Studying computational linguistics and web design is forcing me to grapple with maths and programming. I’ve known for a while that I want to be literate in those fields. The trouble is that I haven’t been taking the time to practice and let my mind wander around maths and programming topics. It’s still All Language All the Time in my head. I’ve been trying to turn my inner monologue on maths and programming. Here are some texts I’ve been using.

Programming and web design

I’ve written about 100 pages of a programming workbook template similar to my language learning workbook template. I’ll post it once it’s presentable. I’m getting ideas for the workbook from resources that f friends and colleagues have recommended, such as the following:

The Odin Project

This website is comprehensive and well-structured. They give us learners everything we need to find information, check our understanding, and test ourselves. At the same time, the materials are organized so that we’re completely responsible for making the most of all the available resources. And after finishing all of the coursework, we can point to specific useful things we’ve made.

I particularly appreciate the way that the practice problems are designed to make learners feel lost, yet there are resources available to guide us when we get stuck. I’m not used to not knowing how to solve a problem. These days I rarely encounter problems that completely stump me. I don’t always get the right answer, but I rarely have the feeling of being totally lost. I’ll try one technique, take it is far as I can, and if I hit a dead end, start over and try again with something else. On the other hand, if I see a maths or programming problem, I don’t even know where to start. I know that’s natural, but it’s been harder for me to overcome the temptation to not even try.

TkDocs

For some reason, I’m intimidated by GUIs, even though I remember making them with Java in high school and thinking they weren’t that bad. I like this tutorial because shows how to do the same thing in a few different languages.

Programming course websites:

There are so many!

Natural language processing

Natural Language Processing with Python by Steven Bird, Ewan Klein, and Edward Loper

This was the first text I found on the subject. I’m about halfway through it now. It’s really clear.

Foundations of Statistical Natural Language Processing by Chris Manning and Hinrich Schütze

This was the second text I started reading after Natural Language Processing with Python, and it’s *way* more challenging. When I started reading, I was totally lost by the first exercise. I looked at the problems and I didn’t even understand what they were asking. Fortunately, It’s starting to make more sense now that I’m reading the Jurafsky and Martin text, and now that I’ve found a more accessible introduction to set theory and probability.

Speech and Language Processing by Daniel Jurafsky and James Martin

This is the text that’s really got me interested in natural language processing. It’s much more technical than Natural Language Processing with Python, but the writing is superb. I’m hoping that after I finish this text, I’ll be able to understand Foundations of Statistical Natural Language Processing without feeling completely hopeless.

Maths

A Probability Course for the Actuaries: A Preparation for Exam P/1 from Arkansas Tech University

SticiGui from UC Berkeley.

I did just fine in my middle and high school maths classes, but I never really had a decent conceptual understanding of maths. It’s caught up with me,  as Daniel Willingham has warned it will. So I really need to start over. After a few months of banging my head against a wall trying to understand the statistics and probability sections in Foundations of Statistical Natural Language Processing, feeling like a hopeless idiot, I found the two resources above. The formulas mostly make sense now. It’s a huge relief.

A First Course in Abstract Algebra by John B. Fraleigh

My mathematician friend lent me this book. I enjoy it, and it’s very clear. I wish I had known earlier what university-level maths is all about. I would’ve stuck with it for longer.

My favorite part about this text is that I’m no longer intimidated by jargon. I’ve always been more comfortable with linguistic jargon than I am with maths or programming jargon. I know it’s irrational: ‘morphotactics’ and ‘semantic field’ don’t faze me at all, so why should ‘commutative ring’ and ‘finite state tranducer’ make me nervous?

All those terms describe things or ideas that are worth thinking and talking about as a group, so it helps to give those collections a name. Fraleigh’s text is very accessible. The definitions are clear, the steps are explicit, and for the most part he avoids phrases such as “obviously” and “by an elementary algebra technique” which make me want to throw the book across the room.

Languages

I haven’t given up on learning languages, though. I’m still going to be using and updating my language learning workbook template. I’m working towards using the ideas in that template to make an online resource for learning and practicing languages. I’ve been meaning to be more active on lang-8, and now I’ve got lots of things to potentially write about. I want to improve my written Chinese, so I’m working through a few books from the 别怕作文 series. As for Spanish, French, and German, Reverso is a fantastic resource.

Why I’m sceptical of polyglots

I don’t mean that I think polyglots are trying to trick people. If you watch Tim Doner’s very interesting TED Talk, you’ll notice he’s very careful about describing his proficiency and the media hyperbole that surrounded his story. Judith Meyer also describes her experience modestly and realistically.

When I say “sceptical,”  I mean that I don’t think that polyglots actually know languages as well as we want to believe they do.

If I want to get good at a language, I mean really good, I have to follow an aggressive media consumption routine every day:

  • 1 book review
  • 1 music review
  • 1 article from each section( ‘society’, ‘life’, ‘culture’, ‘education’, ‘health’, ‘science’, ‘international’, ‘local’, ‘opinion’) of at least 1 major news source.
  • 1 article from news magazines that have me as their target audience.
  • 1 article from an educational journal
  • 1 article from a linguistics journal
  • 1 wikipedia page each from math, science, history, literature, video games, music, film, TV
  • At least 25-50 pages from a novel
  • At least 3 songs I’ve never heard before.
  • Whatever else I want to read and watch

One of the most challenging aspects of this routine is that I can’t just rush through each video or article. I actually have to want to think about what I’m reading. I actually have to make it a part of my life. Savor it. Play with it in my mind. Make connections between what I’ve just read and what I already know. Ask and answer comprehension questions. Have extended conversations with people, or record myself discussing, or write a thoughtful and detailed journal entry.

And while I’m doing all this, I have to constantly be on the lookout for useful vocabulary. When I find useful vocabulary, I can’t just add it to a notebook or spreadsheet. I have to borrow and apply it, not just to summarize the article where I originally found the vocabulary, but also to other, unrelated articles.

And, on top of whatever I read on a particular day, I have to be sure to go back and revisit what I’ve learned before. Not just reread. Really think about what I thought about the day/week/month before. I don’t even see results for a week or more. Because if I don’t recycle what I learned before, after a few days or weeks, it’ll be like I never learned it. On the other hand, if I think about something every day for a week or two, I’ll probably never forget it.

That’s my process. It sounds difficult and time-consuming. It is. I fail most of the time. I’ve never done it for more than 2 days in a row. I don’t have time, and to be honest, I often don’t have the inclination either. I don’t know many people who’ve done it. Khatzumoto and Olle Linge are the only language learning bloggers who I can imagine have a similar routine. And they each focus on a limited number of languages–I don’t know if they’d be interested in identifying as polyglots.

It’s a challenging routine that requires tremendous discipline, and means that I have less time for other things. This is one reason why people think immersion works. If you put yourself in an environment where the people around you are reading and watching lots of stuff, you’re more likely to feel good about putting in the time to do that. It’s easier to speak and write if you know that you’re communicating with people who care about your opinions and whose opinions are important to you. I don’t think immersion is necessary or sufficient for creating that kind of environment, but it could certainly help. As a friend recently pointed out to me, it’s hard to motivate yourself to write something if you know that you’re the only one who’s going to read it.

This process is also related to why we often equate native speakers with articulate speakers. The most articulate English speakers, for example, have used standard English in many aspects of their daily lives over an extended period. They’ve consumed and discussed tons of media every day for a long time. That’s why they’re articulate in English, not because of some ethnic or cultural advantage. Incidentally, depending on the community and the language, identifying as a native speaker of a language doesn’t guarantee that they have the experience required to be literate and articulate in the language.

So what does this have to do with being suspicious of polyglots? Because this process is necessary for being good at a language, and it’s also really difficult and time-consuming. I’m pretty good at it, but I can barely keep up in even one of the many languages I’ve studied. At the risk of sounding immodest, I spend more time learning languages than anyone else I know, and I have some advantages over other learners. I read, learn, recognize and apply language patterns very quickly, and I remember what I’ve learned longer than most people who I know, even if I haven’t thought about it for a long time. I often forget, but after 2-3 seconds of review, it’s as though I’ve remembered all along.

And still, I can barely stay on top of the work.

In Babel No More, Michael Erard describes several stories of polyglots whose success depends on being able to keep the conversation to a limited range of topics. My German Language Challenge was about learning the minimum required to have a basic conversation about a range of familiar topics—in other words, to fake it—and even then, I haven’t been able to make time for it. So I wonder whether polyglots really have the time, energy, and inclination to get really good at the languages they study.

Questions you should ask a language teacher

Here are some questions you can ask if you really want to understand how a language teacher thinks. Do you have any suggestions for other questions to ask?

  1. What are the teacher’s responsibilities in a language class? What are the student’s responsibilities?
  2. What is worth assessing in a language class? How should it be assessed?
  3. How do you train beginning students to express complex ideas with simple language?
  4. How do you train advanced students to use increasingly precise vocabulary?
  5. How do you diagnose and solve language learning challenges that students face?
  6. How do you choose which errors to fix and which to ignore?
  7. How do you distinguish language rules, prescriptive norms, and personal preference?
  8. How do you create a safe environment for failure?
  9. How do you encourage students to make mistakes and learn from them?
  10. How do you train students to give and respond to feedback?
  11. How do you encourage students to think deeply about the knowledge and skills you teach? How do you recycle material so that students think deeply about it over an extended period?
  12. How do you encourage students to be aware of register, audience, and purpose in their work?
  13. What misconceptions do people have about language and learning? How do you resolve those misconceptions?
  14. How do you deal with persistent errors in student work?
  15. How do you engage students for whom your class is too difficult or too easy?
  16. How do you help unmotivated students learn?
  17. What challenges do students face when they practice? How do you help students overcome those challenges? How do you overcome those challenges in your language learning?
  18. What do good language learners do? How do you do those things in your daily life? How do you encourage learners to do those things?
  19. What does ultimate success looks like for a language learner?
  20. What is the role of the language in your daily life? What communities do you explore?
  21. How do you increase your language skill in your daily life? What challenges do you face every day?
  22. What kinds of texts do you explore in your daily life? How do you share those with students? How would you introduce them to beginning students so that they can appreciate those texts?
  23. In your daily life, how do you engage with the knowledge and skills that you demand of your students?
  24. What are some differences between beginning, intermediate, and advanced language learners?
  25. What are some differences between language learning in elementary school, middle school, high school, college, and after college?
  26. What does ‘culture’ mean in a language classroom? What does it mean for a student to demonstrate intercultural competence? How do you teach and assess that?
  27. How do you expose students to diversity, discussion, and disagreement in the communities that use the language?

I failed. Time to start over.

The language challenge: Complete my Language Learning Challenge workbook, learning as much of the language as I can by myself, using freely-available resources, and not getting any outside feedback or corrections.

I really thought I was going to be diligent about going through with the language challenge. But most of the time, I wasn’t. I went for long stretches without doing anything.

I also meant to read/watch something interesting every day in Mandarin, Spanish, and Vietnamese, then write/talk about my thoughts. But most of the time, I didn’t. I don’t think I ever went two days in a row doing that, and I’ve been telling myself for a long time that I would.

When I say that I failed, I don’t mean that I didn’t learn anything. I know plenty about German now. When I review my work, the information comes back to me, and I can quickly regain what I’ve lost. But overall, I failed.

I don’t mind admitting that I failed, because most language learners fail. I think this is why people still take language classes even though there are so many free resources available. Honestly, you can learn pretty much any language you want for free. At the same time, this is my hobby and I’m quite good at it, but even still, it’s still hard for me to force myself to do the things I need to do. It’s much easier to do the things that will make you learn if you have a teacher guiding you and holding you accountable–that is, assuming that the teacher understands how learning works and makes you do the things that will cause you to learn. But that’s a separate issue that I’ll talk about in another post.

So, I failed. Oh, well. It’s time to start again. I don’t think I need to change the parameters of the language challenge. I just need to actually do the things I originally set out to do.

Here are some things I learned:

1. It’s easy to fool myself into thinking I’ve learned when I haven’t.

A few weeks ago, someone introduced me to the work of Daniel Willingham and Stephen L. Chew.  They each say a few things which really resonate with me. I hope I haven’t misrepresented them:

  • The most effective way to see if you know something is to practice using the information the way you will actually have to use it.
  • We remember what we think deeply and frequently about.
  • Effective study strategies reward us for thinking deeply and making connections between concepts.
  • Ineffective study strategies reward us for being able to recall information in a superficial way.

Why does this matter? The only real test of language ability is being able to say and write interesting things. That’s it. If I’m not doing that, I’m not really practicing the language. Copying into the workbook is not a test. Flashcards are not a real test. Matching, multiple-choice, true-false, and fill-in-the-blank quizzes are not real tests. Reviewing my notes and thinking ‘that looks familiar’ is not a real test.

I’m not saying those things aren’t useful, but they are overused at the expense of real learning. They’re easy, satisfying, and not cognitively demanding at all. For most learners, including myself, it would be better to cut them out entirely for a while, and then maybe reintroduce them gradually, in small amounts, after already having developed the habit of practicing how to say and write interesting things.

What I will do differently: I’ve updated my workbook template to include sample writing and speaking prompts. I’ll start each study session with a writing/speaking prompt instead of starting with workbook prompts. I’m going to try not using flashcards for a while.

2. The resistance.

During the language challenge, I knew what I had to do. So why didn’t I do the one thing I knew I needed to do? I learned that I’m not the only one who’s been asking myself about that. Writers and bloggers talk about ‘the resistance.’ This post from The Sarcastic Muse describes the feeling very well and offers a long list of useful strategies. Here are some observations that have worked for me:

  • Creating an outline beforehand helps.
  • It’s OK if the first draft is really bad.
  • It’s OK if the first draft doesn’t include any new vocabulary.

What I will do differently: in addition to the writing and speaking prompts, I’ve created this guide to the steps I should follow when I practice speaking and writing. Following those steps helps me face ‘the resistance’ head-on.

3. It takes time

Language-learning guides often suggest that practicing for 10-15 minutes per day is better than practicing for 2 hours once per week. This is probably true, and a little bit of something is better than a whole lot of nothing, but I’m not sure whether 10-15 minutes is enough learn something useful, think about it deeply, and practice it in a meaningful way. My experience so far has been that anything worth creating in a language takes me at least 60 minutes. I’m very comfortable writing in English, and yet I spent about 3 hours on this blog post, and I will probably make more adjustments later.

4. The topics are all wrong.

Language courses and textbooks follow a fairly predictable pattern: start with greetings, then exchanging personal details, then talking about family, personality, appearance, etc.. Why?

Part of the reason that people fail to learn a language is because they don’t practice writing and talking about things they actually care about. Some classes force students to think at a level which is a lot more superficial than they’re capable of. That also explains the phenomenon of people who succeed in language classes without actually being able to communicate. I’ve seen courses that talk about greetings and polite expressions for 3 or 4 weeks. I’ve seen a 4-week long ‘unit’ about fruit. How tedious.

Teachers also have a lot of trouble finding interesting texts about the topics we’re covering. Maybe we’re thinking about it the wrong way. If I have to try so hard to find examples of texts about that topic, maybe I’ve chosen the wrong topic.

We spend most of our time expressing ourselves about things that have happened in our day, new things we learned, what we’ve observed in media, and what we think about familiar and unfamiliar issues. We agree, disagree, explain, solve misunderstandings. We use our words to make people think and feel things. So why not practice using a limited vocabulary to express those important ideas? ‘Complex’ doesn’t necessary have to mean ‘difficult.’

If I want to learn a language, all I need are:

  • texts about interesting things that I actually want to speak and write about
  • resources to make those texts comprehensible
  • texts about how the language works.

For German, everything’s available online: it’s easy to find people talking and writing in German about every imaginable subject. There are fantastic, free, and comprehensive grammar guides. I can use corpus-based dictionaries such as Reverso to learn to say whatever I want if I can’t find it in the texts I’m reading.

What I will do differently: I’ve updated the workbook and spreadsheet templates to include the 40 sentence stems and 140 words that I use most often. I’ve also created a guide for paraphrasing strategies, which I’ve linked to in the workbook. This list may be slightly different for different people, depending on individual mannerisms. When I start a language, I will learn that vocabulary first, and then get right into using the vocabulary to write and talk about things I actually want to talk about. From now on, when I start a new section of the workbook, I will start with a text about the topic. If I can’t find a text that’s relevant and interesting, I won’t study that topic.

German language challenge: Was ist Kultur?

The language challenge: Complete my Language Learning Challenge workbook, learning as much of the language as I can by myself, using freely-available resources, and not getting any outside feedback or corrections.

I’m going to start posting the things I write. I will edit and correct my own writing as I learn more. I want to see if I can fix my own mistakes without anyone giving me feedback.

17 Oct 21:56 made outline: just some boxes with lines connecting them.

17 Oct 21:57 started writing based on what I put in the outline.

18 Oct 23:17 Finished first draft: 200 words. Once again, it took me about an hour to write 200 words. Time to add more of the the critical sentence stems.

18 Oct 23:53 Added critical sentence stems, essential words, and some emotions.

18 Oct 0:21 Added more essential words and some phrases from Reverso “Context” mode.

Wenn man über ein Land und sein Volk spricht, hören wir sehr oft dieses Wort: “Kultur”. Viele Leute sagen, dass wir Kultur eines Landes kennen müssen, wenn wir diese Sprache lernen wollen. Wir sagen dass es genug ist, und das klingt uns richtig. Am Anfang habe ich auch so gedacht. Aber können wir noch etwas mehr darüber denken? Was ist Kultur? Es gibt viele Bücher darüber, und auch schon viele nützliche Webseiten. Ich werde ein bisschen nervös wenn ich darüber spreche, weil ich weiß, dass manche Leute starke Gefühle für dieses Thema haben, und daher habe ich Sorge, dass diese Leute sauer werden, wenn ich etwas sage, das ihnen nicht gefällt. Wenn es um Sprachen und Kultur geht, sage ich viele Dingen, die anderen nicht gefällt.

Auf jeden Fall, es gibt hier ein Brochure über Kultur. Und hier ein Slideshow, und auch ein Aufsatz.

Ich habe auch ein Video gefunden. Es gibt manche Wörter und Sätze die ich nicht lesen konnte…

Diese Aufsätze haben gefragt: Was ist Kultur? Es meint viele verschiedene Dinge für viele verschiedene Menschen, zum Beispiel:

Hochkultur: Literatur, Kunst, Küche, Musik, Theater

Ideen: Glauben, Religion, Philosophie, Ethik, Etikette, Tabus

Informationen: Geschichte, Mythen, Bezügen zur Popkultur oder Literatur, Sprichwörter, Berühmtheiten

Artefakte: Relikte, Trachtenkleid

Und etwas später habe auch gelesen:

Viele Menschen denken bei Kultur zuerst an Kunst, Malerei, Musik oder ähnliches. Natürlich sind auch dies Aspekte von Kultur, letztlich stellen sie aber nur einen kleinen Ausschnitt der komplexen kulturellen Wirklichkeit dar.

Es is vielleicht eigentlich nicht wichtig, die richtige bedeutung zu finden. Wir können sagen, was uns in den Sinn kommt. Viel wichtiger ist die Leute kennenzulernen, die diese Sprache sprechen. Welchen Gruppen und Gemeinschaften gehören sie an? Was haben sie gemeinsam? Worüber sind sie sich uneinig? Das ist etwas das wir nicht in einem einzigen Buch finden können. Und in Wirklichkeit können wir das auch nicht von einem einzigen Lehrer oder Freund lernen.

Dog years: why I’m not worried about fossilization

For the German Language Challenge, I’m specifically not getting feedback from anyone else. I want to see if I can learn to fix my own errors. Our intuition suggests that this isn’t possible, but I’m not convinced.

We often hear that language learners will develop persistent bad habits if:

  • They’re allowed to say and write things that are full of errors.
  • Their teacher chooses to ignore some errors.
  • They learn from other learners without feedback from a teacher.
  • They don’t learn the “correct” way to speak and write from the beginning.

Language knowledge is just like any other kind of knowledge that you have to apply accurately and creatively, such as jazz improvisation. By “jazz improvisation,” I mean the cognitive skill of creating the melodies, not the physical technique of playing. Learning a language isn’t really like playing an instrument or a sport, because muscle memory isn’t all that important (maybe except for pronunciation, but that’s a topic for another day).

Anyway, when you’re trying to apply knowledge accurately and creatively, you can only pay attention to so many things at once, so you’ll have some misconceptions at first. The better you get, the more you notice, and you’ll be able to resolve those misconceptions one by one.

Here’s an example:

Imagine that you’re talking to a friend about dogs, and your friend tells you that 1 dog year equals 7 human years. A few days, weeks, or months pass, and you come across this article. You realize you’ve been wrong all along. Are you going to look back with embarrassment at all the times you corrected someone’s calculations of dog years? Probably. Did anyone have to explain to you that you’ve been wrong all this time? No, you read the article on your own and made that connection yourself. Now that you recognize your error, if you have to calculate another dog’s age, you probably won’t go back to the “human years * 7” formula unless you’re forgetful or lazy.

That’s why I’m not worried about making mistakes. When you first start learning a language, like anything else, you do the best you can with the limited information you have. As you study more, you get more information. Sure, you’ll look back and realize that a lot of what you used to say was wrong and/or embarrassing, but so what? That’s one lesson that I learned from studying Modern Standard Arabic. Most Arabic textbooks contain a few sections which say something along the lines of: “strictly speaking, what you’re saying is a little bit wrong, but don’t worry about it for right now.”

So what can people do about the bad habits they’ve developed? When you speak and write in a language, learn how to pay attention to what you’re doing as you’re doing it, and update your knowledge and skills based on what you learn from your environment. You have to think about what you just said and wrote, consider other possibilities, think of possible ways that you might have gone wrong, and then decide what’s best based on what you know. It’s an important skill, but it doesn’t come naturally to most people, and most language teachers don’t teach it, or don’t know how.

German Language Challenge: writing

The language challenge: Complete my Language Learning Challenge workbook, learning as much of the language as I can by myself, using freely-available resources, and not getting any outside feedback or corrections.

I’m going to start posting the things I write. I will edit and correct my own writing as I learn more. I want to see if I can fix my own mistakes without anyone giving me feedback.

16 Oct 9:30 updated workbook. Here’s the new version.

16 Oct 9:44 started food glossary

16 Oct 10:05 finished food glossary

16 Oct 14:11 started 01C

16 Oct 15:01 finished 01c, next task is to describe what I will do tomorrow, using the writing steps I created.

16 Oct 15:05 started graphic organizer

16 Oct 15:07 finished graphic organizer, stared writing what I could, using English for things I didn’t know and couldn’t rephrase.

16 Oct 15:55 found this great advice about usage.

16 Oct 15:55 wrote a first draft, approx. 200 words. Lots of errors and things I’m not sure about so the next steps will be:

  • Check for errors (no feedback from other people though, want to figure it out myself).
  • Add in more essential words and sentence stems
  • Add in more vocabulary from the workbook and vocabulary list.

Was will ich morgen tun? Zuerst will ich in ein neues vegetarisches Restaurant gehen. Es gibt hier in Saigon sehr viele Restaurants, in den man sehr lecker essen kann. Wenn Sie kein Fleisch essen wollen, können Sie noch sehr gut essen. Aber was für mich nicht so einfach ist, ist dass manche Restaurants nicht leicht zu finden sein. Zuerst muss man Restaurants auf einer Website suchen. Die Website die ich gewöhnlich gebrauche heißt Happy Cow.  Da kann man viele gute Restaurants finden. Nachdem Sie haben ein Restaurant gesucht, mussen Sie die Adresse aufschreiben. Google Maps ist sehr nützlich, aber manchmal gibt es unrichtigen Angaben. Zum Beispiel sagt es dass das Restaurant “hier” ist, aber in Wirklichkeit ist sehr weit von jenem Ort. Hoffentlich ist das Restaurant immer noch da, so nehme ich an. Dann will ich auch zwei chinesische Bücher auslesen. Ich will mehr als genauso 50 Seiten pro Tag lesen, aber gewöhnlich habe ich nicht genug Zeit. Es ist leichter für andere, glaube ich. Und zum Schluss muss ich etwas in meinem Blog schreiben. Gewöhnlich schreibe ich über die Geschichte, die ich gelesen habe. Ich denke: was muss ich hier schreiben? Ich weiß nicht. Ich habe schon viele Drafts begonnen aber sie sind noch unvollendet.