Parallel distributed processing is a computational approach used to model cognitive processes using networks of interconnected processing units. It involves breaking down complex processes into simpler components and assigning them to different processing units, which work in parallel to complete the task.