Monthly Archives: October 2024

October 31, 2024 · 9:49 pm

AI in Epi 554 (part 3)

Debugging 2: When the code does not fail, but still is wrong

Reminder of Intro and Debugging 1: AI is BS, but not useless; Prompt with error, get suggestions, keep asking, if necessary.

This time, what if there is no error, but something is wrong?

  1. Create a MCVE (minimal complete verifiable example), possibly including a screenshot
  2. Describe the issue to a ChatBot, and keep iterating if necessary. (This will be necessary more often than for Debugging when Code Fails, because then you have an error message to work with!)

The anatomy of an MCVE (follow link for more detail):

  1. Minimal: Use as little code as possible that still produces the same problem
  2. Complete: Provide all parts someone else needs to reproduce your problem in the question itself
  3. Reproducible: Test the code you’re about to provide to make sure it reproduces the problem

Remember the first precept of prompt engineering:  Tell AI who you want it to imitate e.g. “You are a friendly and expert teaching assistant.”

Lauren Wilner (Epi 560 TA) says: I advise the students to tell ChatGPT something like: “Please pretend to be my TA who I go to when I have trouble with my coding in my Data Management class. I am coding in R, and she helps me by giving me tips on rewriting my code and gives me sample lines of code I can use and ensure they do what I want. She also tells me how to know whether the code that she gave me is doing what I expected. The code I am working does X. Currently, I have the following lines of code: Y. Can you please help me rewrite this into a loop?”

Watch out for a common pitfall with BS Bots! Lauren Wilner says, “I find that if I ask ChatGPT what is wrong with something or how something can be improved or whether something is X, it will never say “looks good” or “no problems” or “no improvements needed” – it will always make some change or update. Sometimes that update makes things worse! So, overuse can be an issue.

“Again, it comes back to knowing what you expect or want to see and using ChatGPT to assist you and not do things for you.”

In summary: AI is BS, not useless; Useful for debugging; don’t stay stuck for long, prompt with MCVE code example, and polite request for help. Keep convo going if necessary. 

Comments Off on AI in Epi 554 (part 3)

Filed under education, Uncategorized

Tagged as , , , ,

October 24, 2024 · 11:14 pm

AI in Epi 554 (part 2)

Following up on the general guidance I offered Epi 554 last week, this week I tried to get specific about how to use AI assistance in debugging. I think there is room for improvement, but I’m going to get it out to you, and maybe you’ll tell me how to improve.

Debugging 1: When the code fails

Maybe I’ve told you that AI is BS.  But that doesn’t make it useless.

Useful for debugging: use it so that you don’t stay stuck for long
example: error in code from Lab 2, shown below:

(if you know how to fix this… don’t worry you’ll have an error msg that is less obvi soon; and if you are above-average in debugging… this approach might make you worse!)

Teach an advanced AI technique called “Prompt Engineering“, example: paste error, type “why?” — aside: be polite in your prompts, for a better world and for better answers.  Let’s not go through the details of the answer in detail — I want to focus on how and when to ask

  • You can be more verbose, e.g. you can explain what you were trying to do, paste your error, and ask why you got this error and if it has ideas on how to fix the error that you got.
  • You can also use the first precept of prompt engineering: tell AI who you want it to be.  e.g. “You are a friendly and expert teaching assistant.” or “You are a busy and distracted professor.” (?) 
  • Customize as preferred, e.g. if appropriate, you can start with “you answer in English, but you know that I speak Spanish as a native and English is not my first language.”

You try: here is an error to work with [[I didn’t actually come up with this]], and an answer that still doesn’t fix it.  What might you ask next?

Lauren Wilner (Epi 560 TA) says: For debugging, I have found that ChatGPT is mediocre. I give it the code I ran and the error I got, generally, but I find that often it gives me either (1) code that has the same error again or (2) new code that has a different error.

Summary: AI is BS, not useless; Useful for debugging; don’t stay stuck for long, prompt with code example, and polite request for help. Keep convo going if necess.  It is just imitating the way words often hang together in online text, like stack overflow and cross validated, but if it gets your code to run… then you have running code!

Comments Off on AI in Epi 554 (part 2)

Filed under education

Tagged as , , , ,

October 18, 2024 · 9:21 pm

AI in Epi 554 (part 1)

I offered students in Introduction To Epidemic Modeling For Infectious Diseases some general guidance on using AI this week, and I thought I’d share it more broadly. We are going to get into specifics for AI-assisted coding over the next few weeks, because that is one area where I think this stuff might really help them in this course.

Introduction to Generative AI in Epi 554

  • Not magic — “just” next word prediction
  • But it is next word prediction so good that it keeps me up at night, like in existential crisis
  • How does it do it?  Statistical language model — a conditional probability distribution
  • This is the platonic ideal of the philosophical notion of “bullshit”
  • BS does not mean useless; it can be useful!  For somethings… but…
  • Studies have found that AI is good for helping people with average-to-poor skills in an area attain slightly-above-average performance — so what are you average-to-poor at???
  • Prof Steve Mooney says: for 554 (and all classes), the only point of doing coursework is to learn what you’re doing.  If you learn to use AI to help you code in general, great!  That’s a useful skill.  BUT if you only learn to use AI to help you complete this coursework, it’s like you just overfit your model – you can’t project forward usefully with it.  So, your proper focus ought to be on how AI helps you avoid busywork/debug faster/more deeply understand what you’re doing, not just how to get done sooner.
  • ChatBots can help you build on your existing skills. Lauren Wilner (Epi 560 TA) says: I find ChatBots useful for things that are at the edge of what I know. That is slightly different, I think, than areas where I have average-to-poor skills.
  • I will be demonstrating its use through ChatBots and not through coding assistants (like GitHub co-pilot); I think this is best matched to this class, but I’m still learning!
  • Take responsibility. Lauren Wilner (Epi 560 TA) says: I know it sounds obvious, but I would emphasize that they need to read and edit what ChatGPT or other AI tools give them. A lot of them seem to skip that step and trust it blindly, and I would strongly remind them that they need to use it for advice and feedback, not for answers.
  • Don’t hide it. Lauren Wilner (Epi 560 TA) says: The students who are the most transparent with me in office hours or elsewhere in terms of where they are stuck, what they asked ChatGPT, and why they are still stuck, are the students that I find to be the most successful. The students who hide their use of AI struggle more because they often don’t really understand what they are doing. Whatever you can do to create a welcoming environment in terms of AI tools but also a cautionary tale that they can’t just give you straight ChatGPT output, the better results I think you will have!
  • Build on what you know. Lauren Wilner (Epi 560 TA) says: Try to use AI to help you build on things you already know. For example, if you are told to write a loop but already know how to do what you want to do by copying and pasting 10 times, write out what you would do and then ask ChatGPT to give you advice on putting it into a loop. That way, you can run your code both using what you wrote and what ChatGPT gave you.

Comments Off on AI in Epi 554 (part 1)

Filed under education

Tagged as