Webbläsaren som du använder stöds inte av denna webbplats. Alla versioner av Internet Explorer stöds inte längre, av oss eller Microsoft (läs mer här: * https://www.microsoft.com/en-us/microsoft-365/windows/end-of-ie-support).

Var god och använd en modern webbläsare för att ta del av denna webbplats, som t.ex. nyaste versioner av Edge, Chrome, Firefox eller Safari osv.

Default user image.

Bo Bernhardsson

Modellering och styrning av osäkra system. Programdirektör för masterprogrammet i maskininlärning, system och styrning.

Default user image.

Dual Control by Reinforcement Learning Using Deep Hyperstate Transition Models

Författare

  • Christian Rosdahl
  • Anton Cervin
  • Bo Bernhardsson

Summary, in English

In dual control, the manipulated variables are used to both regulate the system and identify unknown parameters. The joint probability distribution of the system state and the parameters is known as the hyperstate. The paper proposes a method to perform dual control using a deep reinforcement learning algorithm in combination with a neural network model trained to represent hyperstate transitions. The hyperstate is compactly represented as the parameters of a mixture model that is fitted to Monte Carlo samples of the hyperstate. The representation is used to train a hyperstate transition model, which is used by a standard reinforcement learning algorithm to find a dual control policy. The method is evaluated on a simple nonlinear system, which illustrates a situation where probing is needed, but it can also scale to high-dimensional systems. The method is demonstrated to be able to learn a probing technique that reduces the uncertainty of the hyperstate, resulting in improved control performance.

Avdelning/ar

  • Institutionen för reglerteknik
  • LTH profilområde: AI och digitalisering
  • LTH profilområde: Teknik för hälsa

Publiceringsår

2022

Språk

Engelska

Sidor

395-401

Publikation/Tidskrift/Serie

IFAC-PapersOnLine

Volym

55

Dokumenttyp

Konferensbidrag

Ämne

  • Control Engineering

Nyckelord

  • adaptive control
  • adaptive control by neural networks
  • Bayesian methods
  • nonlinear adaptive control
  • reinforcement learning control
  • stochastic optimal control problems

Conference name

14th IFAC Workshop on Adaptive and Learning Control Systems, ALCOS 2022

Conference date

2022-06-29 - 2022-07-01

Conference place

Casablanca, Morocco

Status

Published

Projekt

  • Efficient Learning of Dynamical Systems

ISBN/ISSN/Övrigt

  • ISSN: 2405-8963