Episodes

  • MLG 001 Introduction
    Feb 1 2017

    Show notes: ocdevel.com/mlg/1. MLG teaches the fundamentals of machine learning and artificial intelligence. It covers intuition, models, math, languages, frameworks, etc. Where your other ML resources provide the trees, I provide the forest. Consider MLG your syllabus, with highly-curated resources for each episode's details at ocdevel.com. Audio is a great supplement during exercise, commute, chores, etc.

    • MLG, Resources Guide
    • Gnothi (podcast project): website, Github
    What is this podcast?
    • "Middle" level overview (deeper than a bird's eye view of machine learning; higher than math equations)
    • No math/programming experience required

    Who is it for

    • Anyone curious about machine learning fundamentals
    • Aspiring machine learning developers

    Why audio?

    • Supplementary content for commute/exercise/chores will help solidify your book/course-work

    What it's not

    • News and Interviews: TWiML and AI, O'Reilly Data Show, Talking machines
    • Misc Topics: Linear Digressions, Data Skeptic, Learning machines 101
    • iTunesU issues

    Planned episodes

    • What is AI/ML: definition, comparison, history
    • Inspiration: automation, singularity, consciousness
    • ML Intuition: learning basics (infer/error/train); supervised/unsupervised/reinforcement; applications
    • Math overview: linear algebra, statistics, calculus
    • Linear models: supervised (regression, classification); unsupervised
    • Parts: regularization, performance evaluation, dimensionality reduction, etc
    • Deep models: neural networks, recurrent neural networks (RNNs), convolutional neural networks (convnets/CNNs)
    • Languages and Frameworks: Python vs R vs Java vs C/C++ vs MATLAB, etc; TensorFlow vs Torch vs Theano vs Spark, etc
    Show More Show Less
    7 mins
  • MLG 002 Difference Between Artificial Intelligence, Machine Learning, Data Science
    Feb 9 2017
    Artificial intelligence is the automation of tasks that require human intelligence, encompassing fields like natural language processing, perception, planning, and robotics, with machine learning emerging as the primary method to recognize patterns in data and make predictions. Data science serves as the overarching discipline that includes artificial intelligence and machine learning, focusing broadly on extracting knowledge and actionable insights from data using scientific and computational methods. Links Notes and resources at ocdevel.com/mlg/2 Try a walking desk - stay healthy & sharp while you learn & codeTrack privacy-first web traffic with OCDevel Analytics. Data Science Overview Data science encompasses any professional role that deals extensively with data, including but not limited to artificial intelligence and machine learning.The data science pipeline includes data ingestion, storage, cleaning (feature engineering), and outputs in data analytics, business intelligence, or machine learning.A data lake aggregates raw data from multiple sources, while a feature store holds cleaned and transformed data, prepared for analysis or model training.Data analysts and business intelligence professionals work primarily with data warehouses to generate human-readable reports, while machine learning engineers use transformed data to build and deploy predictive models.At smaller organizations, one person ("data scientist") may perform all data pipeline roles, whereas at large organizations, each phase may be specialized.Wikipedia: Data Science describes data science as the interdisciplinary field for extracting knowledge and insights from structured and unstructured data. Artificial Intelligence: Definition and Sub-disciplines Artificial intelligence (AI) refers to the theory and development of computer systems capable of performing tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. (Wikipedia: Artificial Intelligence)The AI discipline is divided into subfields: Reasoning and problem solvingKnowledge representation (such as using ontologies or knowledge graphs)Planning (selecting actions in an environment, e.g., chess- or Go-playing bots, self-driving cars)LearningNatural language processing (simulated language, machine translation, chatbots, speech recognition, question answering, summarization)Perception (AI perceives the world with sensors; e.g., cameras, microphones in self-driving cars)Motion and manipulation (robotics, transforming decisions into physical actions via actuators)Social intelligence (AI tuned to human emotions, sentiment analysis, emotion recognition)General intelligence (Artificial General Intelligence, or AGI: a system that generalizes across all domains at or beyond human skill) Applications of AI include autonomous vehicles, medical diagnosis, creating art, proving theorems, playing strategy games, search engines, digital assistants, image recognition, spam filtering, judicial decision prediction, and targeted online advertising.AI has both objective definitions (automation of intellectual tasks) and subjective debates around the threshold for "intelligence."The Turing Test posits that if a human cannot distinguish an AI from another human through conversation, the AI can be considered intelligent.Weak AI targets specific domains, while general AI aspires to domain-independent capability.AlphaGo Movie depicts the use of AI planning and learning in the game of Go. Machine Learning: Within AI Machine learning (ML) is a subdiscipline of AI focused on building models that learn patterns from data and make predictions or decisions. (Wikipedia: Machine Learning)Machine learning involves feeding data (such as spreadsheets of stock prices) into algorithms that detect patterns (learning phase) and generate models, which are then used to predict future outcomes.Although ML started as a distinct subfield, in recent years it has subsumed many of the original AI subdisciplines, becoming the primary approach in areas like natural language processing, computer vision, reasoning, and planning.Deep learning has driven this shift, employing techniques such as neural networks, convolutional networks (image processing), and transformers (language tasks), allowing generalizable solutions across multiple domains.Reinforcement learning, a form of machine learning, enables AI systems to learn sequences of actions in complex environments, such as games or real-world robotics, by maximizing cumulative rewards.Modern unified ML models, such as Google's Pathways and transformer architectures, can now tackle tasks in multiple subdomains (vision, language, decision-making) with a single framework. Data Pipeline and Roles in Data Science Data engineering covers obtaining and storing raw data from various data sources (datasets, databases, streams), aggregating into data lakes, and applying schema or permissions.Feature...
    Show More Show Less
    1 hr and 3 mins
  • MLG 003 Inspiration
    Feb 10 2017

    Try a walking desk to stay healthy while you study or work!

    Show notes at ocdevel.com/mlg/3.

    This episode covers four major philosophical topics related to artificial intelligence. The purpose is to give broader context to why AI matters, before moving into technical details in later episodes.

    1. Economic Automation

    AI is automating not just simple tasks like data entry or tax prep, but also high-skill jobs such as medical diagnostics, surgery, and creative work like design, music, and art. There are two common reactions:

    • Fear: Concern over job displacement, similar to past economic shifts like the agricultural and industrial revolutions.
    • Is your job safe?
    • Optimism: Automation may lead to more comfortable living conditions and economic structures like Universal Basic Income. New job types could emerge, as they have in past transitions.
    2. The Singularity

    The singularity refers to a point of runaway technological growth, where AI becomes capable of improving itself recursively. This concept is tied to "artificial general intelligence" and "seed AI"—systems that not only perform tasks but create better versions of themselves. The idea is that this could trigger extremely rapid change, possibly representing a new phase of evolution beyond humanity.

    3. Consciousness

    I explore whether consciousness can emerge from machines. Since the brain is a physical machine and consciousness arises from it, it's possible that artificial systems could develop similar properties. Related ideas:

    • Qualia: Subjective experiences.
    • Functionalism: If something behaves like it’s conscious, it may be conscious.
    • Turing Test: If a machine is indistinguishable from a human in conversation, it passes the test.
    4. Misaligned Goals and Risk

    I discuss scenarios where AI causes harm not through malevolence but through poorly defined objectives. One example is the "paperclip maximizer" thought experiment, where an AI tasked with maximizing paperclip production might consume all resources to do so. This has led some public figures to raise concerns about AI safety. I don't share the same level of concern, but the topic is worth being aware of.

    References
    • Ray Kurzweil, The Singularity is Near
    • Ray Kurzweil, How to Create a Mind
    • Daniel Dennett, Consciousness Explained
    • Nick Bostrom, Superintelligence
    • The Great Courses, Philosophy of Mind, Brain, Consciousness, and Thinking Machines

    In the next episode, I begin covering the technical foundations of machine learning, starting with supervised, unsupervised, and reinforcement learning.

    Show More Show Less
    18 mins
  • MLG 004 Algorithms - Intuition
    Feb 12 2017
    Every machine learning algorithm works on the same intuition: predict, measure the error, then train by adjusting weights to shrink it. Also covers models, features and online learning through a house price example, plus supervised, unsupervised and reinforcement learning. Links Notes and resources at ocdevel.com/mlg/4 How machine learning algorithms learn: predict, error, train Every machine learning algorithm repeats three steps, and textbooks use two names for each. Predict (infer): make a guess, like a chess move or a house price.Error (loss): an error function scores how far off the guess was.Train (learn): change the model's numbers, called weights, a lot after a big miss and a little after a small one. How to train a model: the house price example Training runs those steps over examples whose answers are known. The example here is house prices in Portland, Oregon, a stock teaching dataset like the Iris flower dataset. Load a spreadsheet of sold houses, say a million rows, into an algorithm such as linear regression.Row by row, it predicts a price, measures the error against the real price and updates its weights.The trained model prices houses it has never seen.When one sells for $30,000 under the prediction, the model retrains on that sale. Retraining on each new example is online learning. Algorithm vs model An algorithm is the code, say a hundred lines of Python, usually with PyTorch (MLG 010 covers languages and frameworks). A model is the algorithm plus the pattern it learned, saved as weights. Features, targets and weights Each spreadsheet row is one house. Each column is a feature: bedrooms, bathrooms, square footage, distance to downtown.The last column, the price, is the target. The algorithm sets it aside and learns to predict it.A feature is numerical (square footage) or categorical, also called nominal (downtown or not; pets allowed: cats, dogs, birds).A weight, a coefficient in math terms, is the number a feature is multiplied by. Training finds the weights, which gives an equation: price = 0.3 × bedrooms + 0.7 × bathrooms + 0.5 × square feet Each row is one such equation and the spreadsheet is a matrix, so solving them together is linear algebra (MLG 008). Types of machine learning algorithms: supervised, unsupervised and reinforcement learning Supervised learning trains on examples that come with the answer, like the houses. It predicts a number (linear regression) or a class such as cat, dog or tree (logistic regression).Unsupervised learning gets no answers. It groups examples by what they have in common without naming the groups, which is clustering. Advertisers use it to segment users by their clicks.Reinforcement learning learns from its own actions toward a goal. A mouse in a maze gets 100 points for the cheese, loses 100 for a pit and loses 1 per step, so it hurries. The episode calls that step cost the floor is lava. From the score alone the algorithm works out the best course of action, called a policy. A common algorithm is the deep Q-network (DQN). MLG 029 covers it. Recommended reading: A Tour of Machine Learning Algorithms and The Master Algorithm A Tour of Machine Learning Algorithms by Jason Brownlee at Machine Learning Mastery lists the main algorithms by learning style and by family.The Master Algorithm by Pedro Domingos is an introductory book on the fundamental algorithms. It comes as an audiobook, for learning while you run, clean or commute.The resources list has the rest of the series' picks. Next: MLG 005, on linear regression. If this episode helped, a rating on Apple Podcasts or Spotify helps other learners find the show.
    Show More Show Less
    22 mins
  • MLG 005 Linear Regression
    Feb 16 2017
    Linear regression is introduced as the foundational supervised learning algorithm for predicting continuous numeric values, using cost estimation of Portland houses as an example. The episode explains the three-step process of machine learning - prediction via a hypothesis function, error calculation with a cost function (mean squared error), and parameter optimization through gradient descent - and details both the univariate linear regression model and its extension to multiple features. Links Notes and resources at ocdevel.com/mlg/5 Try a walking desk - stay healthy & sharp while you learn & code Generate a podcast - use my voice to listen to any AI generated content you want Linear Regression Overview of Machine Learning Structure Machine learning is a branch of artificial intelligence, alongside statistics, operations research, and control theory.Within machine learning, supervised learning involves training with labeled examples and is further divided into classification (predicting discrete classes) and regression (predicting continuous values). Linear Regression and Problem Framing Linear regression is the simplest and most commonly taught supervised learning algorithm for regression problems, where the goal is to predict a continuous number from input features.The episode example focuses on predicting the cost of houses in Portland, using square footage and possibly other features as inputs. The Three Steps of Machine Learning in Linear Regression Machine learning in the context of linear regression follows a standard three-step loop: make a prediction, measure how far off the prediction is, and update the prediction method to reduce mistakes.Predicting uses a hypothesis function (also called objective or estimate) that maps input features to a predicted value. The Hypothesis Function The hypothesis function is a formula that multiplies input features by coefficients (weights) and sums them to make a prediction; in mathematical terms, for one feature, it is: h(x) = theta_1 * x_1 + theta_0 Here, theta_1 is the weight for the feature (e.g., square footage), and theta_0 is the bias (an average baseline). With only one feature, the model tries to fit a straight line to a scatterplot of the input feature versus the actual target value. Bias and Multiple Features The bias term acts as the starting value when all features are zero, representing an average baseline cost.In practice, using only one feature limits accuracy; including more features (like number of bedrooms, bathrooms, location) results in multivariate linear regression: h(x) = theta_0 + theta_1 * x_1 + theta_2 * x_2 + ... for each feature x_n. Visualization and Model Fitting Visualizing the problem involves plotting data points in a scatterplot: feature values on the x-axis, actual prices on the y-axis.The goal is to find the line (in the univariate case) that best fits the data, ideally passing through the "center" of the data cloud. The Cost Function (Mean Squared Error) The cost function, or mean squared error (MSE), measures model performance by averaging squared differences between predictions and actual labels across all training examples.Squaring ensures positive and negative errors do not cancel each other, and dividing by twice the number of examples (2m) simplifies the calculus in the next step. Parameter Learning via Gradient Descent Gradient descent is an iterative algorithm that uses calculus (specifically derivatives) to find the best values for the coefficients (thetas) by minimizing the cost function.The cost function's surface can be imagined as a bowl in three dimensions, where each point represents a set of parameter values and the height represents the error.The algorithm computes the slope at the current set of parameters and takes a proportional step (controlled by the learning rate alpha) toward the direction of the steepest decrease.This process is repeated until reaching the lowest point in the bowl, where error is minimized and the model best fits the data.Training will not produce a perfect zero error in practice, but it will yield the lowest achievable average error for the data given. Extension to Multiple Variables Multivariate linear regression extends all concepts above to datasets with multiple input features, with the same process for making predictions, measuring error, and performing gradient descent.Technical details are essentially the same though visualization becomes complex as the number of features grows. Essential Learning Resources The episode strongly directs listeners to the Andrew Ng course on Coursera as the primary recommended starting point for studying machine learning and gaining practical experience with linear regression and related concepts.
    Show More Show Less
    33 mins
  • MLG 006 Certificates & Degrees
    Feb 17 2017

    People interested in machine learning can choose between self-guided learning, online certification programs such as MOOCs, accredited university degrees, and doctoral research, with industry acceptance and personal goals influencing which path is most appropriate. Industry employers currently prioritize a strong project portfolio over non-accredited certificates, and while master's degrees carry more weight for job applications, PhD programs are primarily suited for research interests rather than industry roles.

    Links
    • Notes and resources at ocdevel.com/mlg/6
    • Try a walking desk - stay healthy & sharp while you learn & code
    Learner Types and Self-Guided Education
    • Individuals interested in machine learning may be hobbyists, aspiring professionals, or scientists wishing to contribute to research in artificial intelligence.
    • Hobbyists can rely on structured resources, including curated syllabi and recommended online materials, to guide their self-motivated studies.
    • The "Andrew Ng Coursera" course is frequently recommended as an initial step for self-learners, and advanced resources such as "Artificial Intelligence: A Modern Approach" and "Deep Learning" textbooks are valuable later.
    MOOCs and Online Certificates
    • MOOCs (Massive Open Online Courses) are widely available from platforms such as Coursera, Udacity, edX, and Khan Academy, but only Coursera and Udacity are commonly recognized for machine learning and data science content.
    • Coursera is typically recommended for individual courses; its specializations are less prominent in professional discussions.
    • Udacity offers both free courses and paid "nano degrees" which include structured mentoring, peer interaction, and project-based learning.
    • Although Udacity certificates demonstrate completion and the development of practical projects, they lack widespread recognition or acceptance from employers.
    • Hiring managers and recruiters consistently emphasize the value of a substantial project portfolio over non-accredited certificates for job-seekers.
    University Degrees and Industry Recognition
    • Master's degrees in machine learning or computer science remain the most respected credentials for job applications in the industry, with requirements often officially listed in job postings.
    • The Georgia Tech OMSCS program provides an accredited, fully online Master's degree in Computer Science at a much lower cost than traditional programs, reportedly leveraging Udacity's course infrastructure.
    • In some cases, a strong portfolio can substitute for formal educational requirements, particularly if the applicant demonstrates practical and scalable machine learning project experience.
    • Portfolio strength is considered analogous to web development hiring, where demonstrated skills and personal projects can compensate for missing degree credentials.
    PhD Pathways and Research Careers
    • A PhD is generally unnecessary for industry positions in machine learning; a master's degree or an exceptional portfolio is usually adequate.
    • Doctoral degrees are most useful for those seeking research roles or wishing to investigate complex theoretical questions in artificial intelligence, rather than working in standard industry applications.
    • PhD programs pay a stipend to students, though the compensation is much less than typical industry salaries, which should factor into an individual's decision-making process.
    Considerations and Resources
    • Choosing an educational path depends on individual goals, available resources, and desired career trajectory; a portfolio of significant machine learning projects is universally beneficial regardless of the chosen route.
    • Community discussions and recruiter perspectives suggest that practical skills, proven through real-world projects, are highly valued in addition to or in place of formal degrees.
    • Interested individuals can review ongoing discussions and perspectives:
      • Self-Guided Data Science Guide (canyon289)
      • Hacker News – Are credentials required?
      • Cole MacLean's Self-Taught AI Blog
      • Hacker News: Self-Study Paths
    Show More Show Less
    15 mins
  • MLG 007 Logistic Regression
    Feb 19 2017

    The logistic regression algorithm is used for classification tasks in supervised machine learning, distinguishing items by class (such as "expensive" or "not expensive") rather than predicting continuous numerical values. Logistic regression applies a sigmoid or logistic function to a linear regression model to generate probabilities, which are then used to assign class labels through a process involving hypothesis prediction, error evaluation with a log likelihood function, and parameter optimization using gradient descent.

    Links
    • Notes and resources at ocdevel.com/mlg/7
    • Try a walking desk - stay healthy & sharp while you learn & code
    Classification versus Regression in Supervised Learning
    • Supervised learning consists of two main tasks: regression and classification.
    • Regression algorithms predict continuous values, while classification algorithms assign classes or categories to data points.
    The Role and Nature of Logistic Regression
    • Logistic regression is a classification algorithm, despite its historically confusing name.
    • The algorithm determines the probability that an input belongs to a specific class, using outputs between zero and one.
    How Logistic Regression Works
    • The process starts by passing inputs through a linear regression function, then applying a logistic (sigmoid) function to produce a probability.
    • For binary classification, results above 0.5 usually indicate a positive class (for example, "expensive"), and results below 0.5 indicate a negative class ("not expensive").
    • Multiclass problems assign probabilities to each class, selecting the class with the highest probability using the arg max function.
    Example Application: Housing Spreadsheet
    • An example uses a spreadsheet of houses with features like square footage and number of bedrooms, labeling each as "expensive" (1) or "not expensive" (0).
    • Logistic regression uses the spreadsheet data to learn the pattern that separates expensive houses from less expensive ones.
    Steps in Logistic Regression
    • The algorithm follows three steps: predict (infer a class), evaluate error (calculate how inaccurate the guesses were), and train (refine the underlying parameters).
    • Predictions are compared to actual data, and the difference (error) is calculated via a log likelihood function, which accounts for how confident the prediction was compared to the true value.
    • Model parameters (theta values) are updated using gradient descent, which iteratively reduces the error by adjusting these values based on the derivative of the error function.
    The Mathematical Foundation
    • The hypothesis function is the sigmoid or logistic function, with the formula: 1 / (1 + e^(-theta^T x)), where theta represents the parameters and x the input features.
    • The error function (cost function) for logistic regression uses log likelihood, aggregating errors over all data points to guide model learning.
    Practical Considerations
    • Logistic regression finds a "decision boundary" on the graph (S-curve) that best separates classes such as "expensive" versus "not expensive."
    • When the architecture requires a proper probability distribution (sum of probabilities equals one), a softmax function is applied to the outputs, but softmax is not covered in this episode.
    Composability in Machine Learning
    • Machine learning architectures are highly compositional, with functions nested within other functions - logistic regression itself is a function of linear regression.
    • This composability underpins more complex systems like neural networks, where each "neuron" can be seen as a logistic regression unit powered by linear regression.
    Building Toward Advanced Topics
    • Understanding logistic and linear regression forms the foundation for approaching advanced areas of machine learning such as deep learning and neural networks.
    • The concepts of prediction, error measurement, and iterative training recur in more sophisticated models.
    Resource Recommendations
    • The episode recommends the Andrew Ng Coursera course for deeper study into these concepts and details, especially for further exploration of multivariate regression and error functions.
    Show More Show Less
    35 mins
  • MLG 008 Math for Machine Learning
    Feb 23 2017
    The math for machine learning is linear algebra, statistics and calculus, and you can learn it through machine learning instead of before it. Also covers matrices, vectors and tensors, probability and inference, gradient descent, convex optimization, and where to learn each. Links Notes and resources at ocdevel.com/mlg/8 What math do you need for machine learning? Linear algebra, statistics and calculus, and you don't have to learn them first. Intro courses like Andrew Ng's open with a primer on the math they use. Go deeper afterward, or give math 20% of your learning time, one day a week. In cooking terms, linear algebra is chopping vegetables, statistics is the cookbook and calculus is the oven. Linear algebra: matrices and vectors Linear algebra is math on matrices and vectors, the easiest of the three. The house spreadsheet from MLG 004 is a matrix, capital X: rows are houses and columns are features. The learned weights, theta (θ), are a vector, a list of numbers. A prediction multiplies each feature by its weight and adds the results: h(x) = θ₀ + θ₁x₁ + θ₂x₂ + θ₃x₃ Nested Python for loops are too slow for a million rows. Linear algebra does it in one operation, run on a CPU or GPU with SIMD (single instruction, multiple data). What is a tensor? A tensor is a list of numbers with any number of dimensions: 0D: a number1D: a vector2D: a matrix3D: a cube, like an image, whose pixels each hold red, green and blue values TensorFlow is named for it, and it's the core data type of PyTorch, now the most used framework (MLG 010). MLA 005 covers tensor shapes. Statistics: probability and inference Statistics supplies the algorithms and is the hardest of the three. Hypothesis functions (linear and logistic regression) and error functions (mean squared error, log likelihood) are statistics formulas, like the logistic regression cost function: cost = −(1/m) × Σ [ y × log(h(x)) + (1 − y) × log(1 − h(x)) ] Probability describes how data is distributed, like prices in a housing market.Inference predicts a new example, like one house, from that distribution. Calculus: derivatives and gradient descent Calculus does the training. The derivative of the loss function is the slope of the error graph at the current θ values, telling each θ which way is downhill and how big a step to take. Repeating that step is gradient descent (MLG 005). Derivatives mostly come from memorized rules, like the power rule and chain rule. Convex optimization Optimization is the branch of math that uses calculus to minimize a function. Convex means the error graph is shaped like a cup. It's often listed as a fourth prerequisite, but you learn it with calculus. Math courses, textbooks and primers Khan Academy, free: Linear algebra, Statistics and probability, plus Calculus 1, Calculus 2 and Multivariable calculus.Textbooks: Introduction to Linear Algebra by Gilbert Strang, All of Statistics by Larry Wasserman and Calculus: Early Transcendentals by James Stewart.Primers from Stanford's CS229: Linear Algebra Review and Reference (26 pages) and Review of Probability Theory (12 pages). The Great Courses: Mathematical Decision Making Mathematical Decision Making: Predictive Models and Optimization is a 24-lecture video course by Scott P. Stevens on operations research: the math of scheduling employees, train departures and factory settings, very close to machine learning's. It's $49.99 to buy, or streams on The Great Courses Plus. The Great Courses also has calculus, statistics and linear algebra courses, in the resources. They're a supplement: the lecturers narrate what they draw, so the videos work as audio. Convert video files to mp3 has the bash script. Next: MLG 009, on deep learning. If this episode helped, a rating on Apple Podcasts or Spotify helps other learners find the show.
    Show More Show Less
    28 mins