ECTS credits ECTS credits: 6
ECTS Hours Rules/Memories Hours of tutorials: 1 Expository Class: 30 Interactive Classroom: 20 Total: 51
Use languages Spanish, Galician
Type: Ordinary Degree Subject RD 1393/2007 - 822/2021
Departments: Electronics and Computing
Areas: Computer Science and Artificial Intelligence
Center Higher Technical Engineering School
Call: First Semester
Teaching: With teaching
Enrolment: Enrollable
The objective of this subject is to equip students with the ability to correctly identify and apply the most appropriate reinforcement learning techniques to address sequential decision problems in various domains. To achieve this, students must understand the theoretical and practical foundations of reinforcement learning models and algorithms, evaluate their strengths and limitations, and determine which approaches are most suitable based on the nature of the environment, the availability of models, the reward structure, and the operational constraints of the problem to be solved.
Topic 1. Introduction to Reinforcement Learning
Topic 2. Modeling the Environment: Markov Decision Processes (MDP)
Topic 3. Value Functions and Bellman Equations
Topic 4. The exploration-exploitation dilemma
Topic 5. Value-based methods
Topic 6. Deep learning in RL: Deep Q-Networks (DQN)
Topic 7. Policy-based methods
Basic bibliography:
- Reinforcement learning notes. Notes will be provided, created by the teaching staff, covering all the topics of the subject.
- Mohit Sewak. Deep Reinforcement Learning. Frontiers in Artificial Intelligence. Springer 2020.
- Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press 2020.
Complementary bibliography:
- Alexander Zan and Brandon Brown. Deep Reinforcement Learning in Action. Manning 2020.
- Laura Graesser and Wah Loon Keng. Foundations of Deep Reinforcement Learning: Theory and Practice in Python. Addison Wesley 2020.
This subject contributes to the following specific competences:
- [CE23] Know the fundamental types of machine learning problems for solving clustering, classification, and prediction tasks.
- [CE24] Know the fundamentals and main techniques of machine learning, to be able to create new models using the most appropriate validation methodology and result presentation for each problem.
- [CE25] Learn the fundamental paradigms and architectures of neural networks, especially deep networks, and be able to recognize the main differences between these paradigms, their algorithmic complexity, and the specific contexts for their use as part of the design of intelligent systems.
- [CE26] The ability to identify the suitability of each machine learning technique for solving a problem, including aspects related to its computational complexity or its explanatory power, in accordance with the requirements established.
Additionally, this subject contributes to achieving the following competences outlined in the Bachelor's degree in Artificial Intelligence:
- Basic: CB2, CB3, CB5
- General: CG3, CG4, CG5
- Transversal: TR1, TR4, TR5
The teaching methodology is aimed at focusing on both the theoretical and practical aspects of reinforcement learning, and how it differs from other types of learning. With this in mind, there are two types of learning activities: theoretical classes and practical classes.
- Theoretical classes. 30 hours of lectures will be given in 1-hour sessions. These are aimed at explaining the concepts underlying this learning paradigm.
- Practical classes. 20 hours of practical classes will be held in the computer lab, in 10 sessions of 2 hours, which will allow students to familiarize themselves from a practical standpoint with the topics covered in the theoretical classes. Attendance in these classes is MANDATORY.
In-person formative activities and their relationship to the program competences:
- Theoretical classes taught by the professor and seminar presentations. Competences addressed: CE23, CE24, CE25, CE26, CG3, CG4.
- Laboratory practical classes, problem-solving, and case studies. Competences addressed: CE23, CE24, CE25, CE26, CB2, CB3, CB5, CG5, TR2, TR4, TR5.
- Exam. Competences addressed: CE23, CE24, CE25, CE26, CG3, CG4, CG5.
Non-presential formative activities and their relationship to the program competences:
- Personal work of the student: consultation of bibliography, independent study, development of scheduled activities, preparation of presentations and assignments. Competences addressed: CE23, CE24, CE25, CE26, CG3, CG4, CG5, TR2, TR4, TR5.
ORDINARY OPPORTUNITY:
Practical work: 40%
Final exam: 60%
The assessment of learning will be based on continuous assessment and a final exam. The subject will include four practical assignments (2.5 points per assignment), corresponding to the content of the subject. Each assignment must be submitted on the specified date and format, and its evaluation will take place through a test. These tests will generally be conducted in the session following the submission.
To pass the subject, and provided that the minimum attendance requirements are met, a grade equal to or higher than 5 (out of 10) must be achieved in both the practical work and the final exam.
EXTRAORDINARY OPPORTUNITY:
Provided that the minimum attendance requirements are met, students may recover failed parts in the ordinary session (practical work and final exam) through an exam.
NON-PRESENTED CONDITION:
Students who have not been assessed in any part of the subject. In addition, students with a score that does not represent more than 10% of the total possible grade of the subject may also choose the Non-Presented condition, provided they inform the course coordinator.
REPEATING STUDENTS:
Assessments from previous years are not carried over.
ATTENDANCE CONTROL:
Attendance at practical sessions is mandatory. If a student attends less than 80% of the practical sessions, they will be considered to have failed the subject.
In case of fraudulent completion of exercises or tests, the university’s academic performance evaluation and qualification review regulations will be applied.
In accordance with the ETSE’s plagiarism policy (approved by the ETSE Board on 19/12/2019), the total or partial copying of any practical or theoretical exercise will result in failure in both opportunities for the course, with a grade of 0.0 in both cases.
In-class work:
- Theory classes: 30 hours
- Practical classes: 20 hours
- Individual tutoring: 1 hour
Total in-class work hours: 51 hours
Personal student work (study, exercises, practicals, projects) and other activities (non-presential evaluation): 99 hours
Due to the strong interrelationship between the theoretical and practical parts, and the progressive nature of the theoretical concepts, it is recommended to dedicate time each day for study or review.
The USC virtual campus will be used for all teaching, publication of materials, practice guidelines, and assignment submissions.
The preferred language for the delivery of theoretical and interactive classes is Galician/Spanish.
Manuel Lama Penin
Coordinador/a- Department
- Electronics and Computing
- Area
- Computer Science and Artificial Intelligence
- Phone
- 881816427
- manuel.lama [at] usc.es
- Category
- Professor: University Professor
Juan Carlos Vidal Aguiar
- Department
- Electronics and Computing
- Area
- Computer Science and Artificial Intelligence
- Phone
- 881816388
- Category
- Professor: University Lecturer
Marcos Fernandez Pichel
- Department
- Electronics and Computing
- Area
- Computer Science and Artificial Intelligence
- marcosfernandez.pichel [at] usc.es
- Category
- Professor: Intern Assistant LOSU
| Tuesday | |||
|---|---|---|---|
| 18:30-20:00 | Grupo /CLIL_02 | Galician | IA.S2 |
| Wednesday | |||
| 17:30-18:30 | Grupo /CLE_01 | Galician | IA.01 |
| 18:30-20:00 | Grupo /CLIL_01 | Galician | IA.01 |
| Thursday | |||
| 15:30-17:00 | Grupo /CLE_01 | Galician | IA.01 |
| 01.08.2026 16:00-20:00 | Grupo /CLE_01 | IA.S1 |
| 01.08.2026 16:00-20:00 | Grupo /CLIL_02 | IA.S1 |
| 01.08.2026 16:00-20:00 | Grupo /CLIL_01 | IA.S1 |
| 06.17.2026 16:00-20:30 | Grupo /CLIL_01 | IA.01 |
| 06.17.2026 16:00-20:30 | Grupo /CLE_01 | IA.01 |
| 06.17.2026 16:00-20:30 | Grupo /CLIL_02 | IA.01 |