<?xml version="1.0" encoding="utf-8"?>
<TEI xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:hal="http://hal.archives-ouvertes.fr/" xmlns:gml="http://www.opengis.net/gml/3.3/" xmlns:gmlce="http://www.opengis.net/gml/3.3/ce" version="1.1" xsi:schemaLocation="http://www.tei-c.org/ns/1.0 http://api.archives-ouvertes.fr/documents/aofr-sword.xsd">
  <teiHeader>
    <fileDesc>
      <titleStmt>
        <title>HAL TEI export of hal-00380870</title>
      </titleStmt>
      <publicationStmt>
        <distributor>CCSD</distributor>
        <availability status="restricted">
          <licence target="https://creativecommons.org/publicdomain/zero/1.0/">CC0 1.0 - Universal</licence>
        </availability>
        <date when="2026-05-17T06:38:25+02:00"/>
      </publicationStmt>
      <sourceDesc>
        <p part="N">HAL API Platform</p>
      </sourceDesc>
    </fileDesc>
  </teiHeader>
  <text>
    <body>
      <listBibl>
        <biblFull>
          <titleStmt>
            <title xml:lang="en">Kalman Temporal Differences: the deterministic case</title>
            <author role="aut">
              <persName>
                <forename type="first">Matthieu</forename>
                <surname>Geist</surname>
              </persName>
              <email type="md5">c025140e3cc06ac39e7de0f0db2a4b70</email>
              <email type="domain">univ-lorraine.fr</email>
              <idno type="idhal" notation="string">matthieu-geist</idno>
              <idno type="idhal" notation="numeric">6945</idno>
              <idno type="halauthorid" notation="string">7492-6945</idno>
              <affiliation ref="#struct-26305"/>
              <affiliation ref="#struct-36484"/>
            </author>
            <author role="aut">
              <persName>
                <forename type="first">Olivier</forename>
                <surname>Pietquin</surname>
              </persName>
              <email type="md5">d7ca8400327f875d5b916c43fee3b2c5</email>
              <email type="domain">univ-lille1.fr</email>
              <idno type="idhal" notation="string">olivier-pietquin</idno>
              <idno type="idhal" notation="numeric">4024</idno>
              <idno type="halauthorid" notation="string">23089-4024</idno>
              <idno type="IDREF">https://www.idref.fr/142821861</idno>
              <idno type="ORCID">https://orcid.org/0000-0002-5386-465X</idno>
              <affiliation ref="#struct-26305"/>
            </author>
            <author role="aut">
              <persName>
                <forename type="first">Gabriel</forename>
                <surname>Fricout</surname>
              </persName>
              <idno type="halauthorid">356858-0</idno>
              <affiliation ref="#struct-36484"/>
            </author>
            <editor role="depositor">
              <persName>
                <forename>Sébastien</forename>
                <surname>Van Luchene</surname>
              </persName>
              <email type="md5">983556412cb0e7fb6d6508a01ac9cf46</email>
              <email type="domain">supelec.fr</email>
            </editor>
          </titleStmt>
          <editionStmt>
            <edition n="v1" type="current">
              <date type="whenSubmitted">2009-05-06 09:44:31</date>
              <date type="whenModified">2023-02-14 03:38:23</date>
              <date type="whenReleased">2009-05-06 09:53:18</date>
              <date type="whenProduced">2009-03-30</date>
              <date type="whenEndEmbargoed">2009-05-06</date>
              <ref type="file" target="https://centralesupelec.hal.science/hal-00380870v1/document">
                <date notBefore="2009-05-06"/>
              </ref>
              <ref type="file" subtype="greenPublisher" n="1" target="https://centralesupelec.hal.science/hal-00380870v1/file/Supelec471.pdf" id="file-380870-572538">
                <date notBefore="2009-05-06"/>
              </ref>
              <ref type="externalLink" target="http://hal.archives-ouvertes.fr/docs/00/38/08/70/PDF/Supelec471.pdf"/>
            </edition>
            <respStmt>
              <resp>contributor</resp>
              <name key="123540">
                <persName>
                  <forename>Sébastien</forename>
                  <surname>Van Luchene</surname>
                </persName>
                <email type="md5">983556412cb0e7fb6d6508a01ac9cf46</email>
                <email type="domain">supelec.fr</email>
              </name>
            </respStmt>
          </editionStmt>
          <publicationStmt>
            <distributor>CCSD</distributor>
            <idno type="halId">hal-00380870</idno>
            <idno type="halUri">https://centralesupelec.hal.science/hal-00380870</idno>
            <idno type="halBibtex">geist:hal-00380870</idno>
            <idno type="halRefHtml">&lt;i&gt;ADPRL 2009&lt;/i&gt;, Mar 2009, Nashville, TN, United States. pp.185-192, &lt;a target="_blank" href="https://dx.doi.org/10.1109/ADPRL.2009.4927543"&gt;&amp;#x27E8;10.1109/ADPRL.2009.4927543&amp;#x27E9;&lt;/a&gt;</idno>
            <idno type="halRef">ADPRL 2009, Mar 2009, Nashville, TN, United States. pp.185-192, &amp;#x27E8;10.1109/ADPRL.2009.4927543&amp;#x27E9;</idno>
            <availability status="restricted">
              <licence target="https://about.hal.science/hal-authorisation-v1/">HAL Authorization<ref corresp="#file-380870-572538"/></licence>
            </availability>
          </publicationStmt>
          <seriesStmt>
            <idno type="stamp" n="SUPELEC">SUPELEC</idno>
            <idno type="stamp" n="CENTRALESUPELEC">Ecole CentraleSupélec</idno>
          </seriesStmt>
          <notesStmt>
            <note type="audience" n="2">International</note>
            <note type="invited" n="0">No</note>
            <note type="popular" n="0">No</note>
            <note type="peer" n="1">Yes</note>
            <note type="proceedings" n="1">Yes</note>
          </notesStmt>
          <sourceDesc>
            <biblStruct>
              <analytic>
                <title xml:lang="en">Kalman Temporal Differences: the deterministic case</title>
                <author role="aut">
                  <persName>
                    <forename type="first">Matthieu</forename>
                    <surname>Geist</surname>
                  </persName>
                  <email type="md5">c025140e3cc06ac39e7de0f0db2a4b70</email>
                  <email type="domain">univ-lorraine.fr</email>
                  <idno type="idhal" notation="string">matthieu-geist</idno>
                  <idno type="idhal" notation="numeric">6945</idno>
                  <idno type="halauthorid" notation="string">7492-6945</idno>
                  <affiliation ref="#struct-26305"/>
                  <affiliation ref="#struct-36484"/>
                </author>
                <author role="aut">
                  <persName>
                    <forename type="first">Olivier</forename>
                    <surname>Pietquin</surname>
                  </persName>
                  <email type="md5">d7ca8400327f875d5b916c43fee3b2c5</email>
                  <email type="domain">univ-lille1.fr</email>
                  <idno type="idhal" notation="string">olivier-pietquin</idno>
                  <idno type="idhal" notation="numeric">4024</idno>
                  <idno type="halauthorid" notation="string">23089-4024</idno>
                  <idno type="IDREF">https://www.idref.fr/142821861</idno>
                  <idno type="ORCID">https://orcid.org/0000-0002-5386-465X</idno>
                  <affiliation ref="#struct-26305"/>
                </author>
                <author role="aut">
                  <persName>
                    <forename type="first">Gabriel</forename>
                    <surname>Fricout</surname>
                  </persName>
                  <idno type="halauthorid">356858-0</idno>
                  <affiliation ref="#struct-36484"/>
                </author>
              </analytic>
              <monogr>
                <title level="m">IEEE International Symposium on Adaptive Dynamic Programming and Reinforcement Learning</title>
                <meeting>
                  <title>ADPRL 2009</title>
                  <date type="start">2009-03-30</date>
                  <date type="end">2009-04-02</date>
                  <settlement>Nashville, TN</settlement>
                  <country key="US">United States</country>
                </meeting>
                <imprint>
                  <biblScope unit="pp">185-192</biblScope>
                  <date type="datePub">2009-04</date>
                </imprint>
              </monogr>
              <idno type="doi">10.1109/ADPRL.2009.4927543</idno>
            </biblStruct>
          </sourceDesc>
          <profileDesc>
            <langUsage>
              <language ident="en">English</language>
            </langUsage>
            <textClass>
              <classCode scheme="halDomain" n="spi.signal">Engineering Sciences [physics]/Signal and Image processing</classCode>
              <classCode scheme="halDomain" n="info.info-ts">Computer Science [cs]/Signal and Image Processing</classCode>
              <classCode scheme="halTypology" n="COMM">Conference papers</classCode>
              <classCode scheme="halOldTypology" n="COMM">Conference papers</classCode>
              <classCode scheme="halTreeTypology" n="COMM">Conference papers</classCode>
            </textClass>
            <abstract xml:lang="en">
              <p>This paper deals with value function and $Q$-function approximation in deterministic Markovian decision processes. A general statistical framework based on the Kalman filtering paradigm is introduced. Its principle is to adopt a parametric representation of the value function, to model the associated parameter vector as a random variable and to minimize the mean-squared error of the parameters conditioned on past observed transitions. From this general framework, which will be called Kalman Temporal Differences (KTD), and using an approximation scheme called the unscented transform, a family of algorithms is derived, namely KTD-V, KTD-SARSA and KTD-Q, which aim respectively at estimating the value function of a given policy, the $Q$-function of a given policy and the optimal $Q$-function. The proposed approach holds for linear and nonlinear parameterization. This framework is discussed and potential advantages and shortcomings are highlighted.</p>
            </abstract>
          </profileDesc>
        </biblFull>
      </listBibl>
    </body>
    <back>
      <listOrg type="structures">
        <org type="laboratory" xml:id="struct-26305" status="OLD">
          <orgName>SUPELEC-Campus Metz</orgName>
          <date type="start">1985-01-01</date>
          <date type="end">2014-12-31</date>
          <desc>
            <address>
              <addrLine>2 rue Edouard Belin 57070 Metz</addrLine>
              <country key="FR"/>
            </address>
            <ref type="url">http://www.metz.supelec.fr/metz/</ref>
          </desc>
          <listRelation>
            <relation active="#struct-303397" type="direct"/>
          </listRelation>
        </org>
        <org type="laboratory" xml:id="struct-36484" status="VALID">
          <orgName>ArcelorMittal Maizières Research SA</orgName>
          <desc>
            <address>
              <addrLine>Voie romaine, BP 30320, 57283 Maizières-lès-Metz</addrLine>
              <country key="FR"/>
            </address>
          </desc>
          <listRelation>
            <relation active="#struct-508203" type="direct"/>
          </listRelation>
        </org>
        <org type="institution" xml:id="struct-303397" status="VALID">
          <orgName>Ecole Supérieure d'Electricité - SUPELEC (FRANCE)</orgName>
          <desc>
            <address>
              <country key="FR"/>
            </address>
          </desc>
        </org>
        <org type="institution" xml:id="struct-508203" status="VALID">
          <orgName>ArcelorMittal</orgName>
          <desc>
            <address>
              <country key="LU"/>
            </address>
          </desc>
        </org>
      </listOrg>
    </back>
  </text>
</TEI>