{"entities":{"Q584085":{"pageid":585852,"ns":120,"title":"Item:Q584085","lastrevid":42595203,"modified":"2025-07-04T11:37:30Z","type":"item","id":"Q584085","labels":{"en":{"language":"en","value":"Solving infinite horizon discounted Markov decision process problems for a range of discount factors"}},"descriptions":{"en":{"language":"en","value":"scientific article; zbMATH DE number 4133862"}},"aliases":{},"claims":{"P31":[{"mainsnak":{"snaktype":"value","property":"P31","hash":"fd5912e4dab4b881a8eb0eb27e7893fef55176ad","datavalue":{"value":{"entity-type":"item","numeric-id":56887,"id":"Q56887"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$67D403C5-1AA0-4F19-AAE4-A802B8DC2E0D","rank":"normal"}],"P159":[{"mainsnak":{"snaktype":"value","property":"P159","hash":"7313f2c6dce52d4c4376089631840af25924c52f","datavalue":{"value":{"text":"Solving infinite horizon discounted Markov decision process problems for a range of discount factors","language":"en"},"type":"monolingualtext"},"datatype":"monolingualtext"},"type":"statement","id":"Q584085$8167BC95-C421-4680-8B0F-00C79C563D8F","rank":"normal"}],"P225":[{"mainsnak":{"snaktype":"value","property":"P225","hash":"b2dc95e03bcf17da9d6f23516cd854f8e8be9ab1","datavalue":{"value":"0692.90101","type":"string"},"datatype":"external-id"},"type":"statement","id":"Q584085$A5E095E7-1A4B-4E01-8266-A942848D82BA","rank":"normal"}],"P27":[{"mainsnak":{"snaktype":"value","property":"P27","hash":"99abadf17b3c3948d7c3bcb851e0aef2fa45bbcc","datavalue":{"value":"10.1016/0022-247X(89)90179-0","type":"string"},"datatype":"external-id"},"type":"statement","id":"Q584085$A0EE7C23-485A-491B-93A8-9FFD8C76D25C","rank":"normal"}],"P200":[{"mainsnak":{"snaktype":"value","property":"P200","hash":"c5ad7b9baf8e531c6b1ad954eb5f2899cc8fe5e5","datavalue":{"value":{"entity-type":"item","numeric-id":85344,"id":"Q85344"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$10795639-FCCD-41C7-B0AA-BA2E9FA33334","rank":"normal"}],"P28":[{"mainsnak":{"snaktype":"value","property":"P28","hash":"7211ad5ca16eb0d22cd0051fff3d0f3af254ceb6","datavalue":{"value":{"time":"+1989-00-00T00:00:00Z","timezone":0,"before":0,"after":0,"precision":9,"calendarmodel":"http://www.wikidata.org/entity/Q1985727"},"type":"time"},"datatype":"time"},"type":"statement","id":"Q584085$DD2C9E09-1D70-47EF-BD73-91E2F2BBD063","rank":"normal"}],"P1448":[{"mainsnak":{"snaktype":"value","property":"P1448","hash":"242a6eab56e3accdc3c4366bbb8aa651d659beff","datavalue":{"value":"Consider the following decision problem. There is a finite set I. For each \\(i\\in I\\) there is a finite action set K(i). If at a decision epoch the state is \\(i\\in I\\) and an action \\(k\\in K(i)\\) is taken then the new state j is a random variable described by a given transition probability p(i,k;j). There is an immediate reward r(i,k) with \\(0\\leq r(i,k)\\leq M<\\infty\\) and discount factor of the shape \\(\\tau =t\\rho\\) where \\(t\\in [0,1]\\) and \\(0\\leq \\rho <1\\). The number t is treated as a parameter.    The paper deals with maximizing the infinite horizon discounted rewards over the policies of the form \\(\\pi =(\\delta)^{\\infty}\\), where \\(\\delta\\) : \\(I\\to \\cup_{i\\in I}K(i).\\)    Let \\(v_ t(i)\\), \\(i\\in I\\) denote infinite horizon expected, discounted reward corresponding to parameter t. The problems are the following:   (a) to find approximations for \\(v_ t\\) over the range [0,1];    (b) to find approximations for \\(v_{t+\\delta}\\) when \\(v_ t\\) is given and \\(\\delta\\) may take value in [0,1-t].    Some algorithms for solving the problems are presented.","type":"string"},"datatype":"string"},"type":"statement","id":"Q584085$E6CC1AA7-36EF-4DDD-85E8-EACC3F49AA9A","rank":"normal"}],"P226":[{"mainsnak":{"snaktype":"value","property":"P226","hash":"377d3ab03372cff12915e0de0374438ff70c3716","datavalue":{"value":"90C40","type":"string"},"datatype":"external-id"},"type":"statement","id":"Q584085$4A42248D-9036-4D4A-9086-E4CDD7A76FB0","rank":"normal"}],"P1451":[{"mainsnak":{"snaktype":"value","property":"P1451","hash":"791c8b0d2ddd33af58578191410e24d8156b8e94","datavalue":{"value":"4133862","type":"string"},"datatype":"external-id"},"type":"statement","id":"Q584085$90905A9A-4874-4243-9AC4-54CF9357C3FB","rank":"normal"}],"P1450":[{"mainsnak":{"snaktype":"value","property":"P1450","hash":"6261db6cc6726dbb5741721deded9bf5e77f85e3","datavalue":{"value":"infinite horizon discounted rewards","type":"string"},"datatype":"string"},"type":"statement","id":"Q584085$91CE03C3-CC80-4771-9BE3-B1833C9D47CE","rank":"normal"}],"P16":[{"mainsnak":{"snaktype":"value","property":"P16","hash":"14fb6158c2e7a30d22594a5ac8edc4708d1a10b2","datavalue":{"value":{"entity-type":"item","numeric-id":181183,"id":"Q181183"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$95D8CF26-7B64-4CE2-8222-C8D122E86F00","rank":"normal"}],"P1447":[{"mainsnak":{"snaktype":"value","property":"P1447","hash":"a059546f0b0486b6a2bbe72db9c16869dfb00d36","datavalue":{"value":{"entity-type":"item","numeric-id":1405193,"id":"Q1405193"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$FC961CC6-51ED-4BA2-A801-AD0BD8751E1F","rank":"normal"}],"P1460":[{"mainsnak":{"snaktype":"value","property":"P1460","hash":"57f7fea50d2ce1b39b695c4a1313582eed405e38","datavalue":{"value":{"entity-type":"item","numeric-id":5976449,"id":"Q5976449"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$3B6E48AD-7B4A-4616-9363-1E72B5A365EF","rank":"normal"}],"P223":[{"mainsnak":{"snaktype":"value","property":"P223","hash":"f9595afdcd6f0a34d637d70f3c89fca691c6293c","datavalue":{"value":{"entity-type":"item","numeric-id":3241504,"id":"Q3241504"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$9FE5B61D-72FF-433F-B4EC-D4111D5B8126","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"7f879498092ca8c11a1a640843409581e3f2645b","datavalue":{"value":{"entity-type":"item","numeric-id":5640967,"id":"Q5640967"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$14B2FAB9-70CC-4886-85DC-1EB965BB8FDC","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"337aa316ba6d56cebc6edab3bc1974fdc76b8872","datavalue":{"value":{"entity-type":"item","numeric-id":5515019,"id":"Q5515019"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$CA544872-5497-49FB-BD98-599B3396F496","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"a1fba1df6d081853d77e27e7fa2b7f904d1ec2e9","datavalue":{"value":{"entity-type":"item","numeric-id":3912356,"id":"Q3912356"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$B47DF39E-E9A1-4C7B-9170-A3E4DFD7F7CC","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"10fbbf5632b969e460b4e76f34636c9efa2fe06b","datavalue":{"value":{"entity-type":"item","numeric-id":3716853,"id":"Q3716853"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$95FE7515-B559-441E-ACA8-E0385DEF6FB7","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"793dbe6731e9183a19ff1e0e74a596df65c7e4ff","datavalue":{"value":{"entity-type":"item","numeric-id":1821705,"id":"Q1821705"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$23E1588F-E0E1-44F9-8CF1-C9C9E9767F1E","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"9039c63560118f49c65f860b1a4c794c8e620067","datavalue":{"value":{"entity-type":"item","numeric-id":3867541,"id":"Q3867541"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$C0F1CE04-4BC7-4D02-99BE-4A756639665D","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"b6ced350c756785569818135335e90ea10efbd00","datavalue":{"value":{"entity-type":"item","numeric-id":3934167,"id":"Q3934167"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$33D4EB7D-8675-4F19-990E-4462AA437A15","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P223","hash":"8d0c83891471dda42666bed3037f7d6a19c11e2d","datavalue":{"value":{"entity-type":"item","numeric-id":3856450,"id":"Q3856450"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q584085$AC4D0E9A-A58F-4709-81BC-888EAEB048C1","rank":"normal"}],"P1643":[{"mainsnak":{"snaktype":"value","property":"P1643","hash":"ecf5ef0a2ab47321f18652690bf5746f09c938df","datavalue":{"value":{"entity-type":"item","numeric-id":1821705,"id":"Q1821705"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"b4f6cf5f30cd553b7e3b7d709048b88b1e12fa73","datavalue":{"value":{"amount":"+0.94214123","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$88E52E3D-0869-4AD9-81C9-C50675ADAE0C","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"9a3f97dfa713cd155144662a9e12be67d0f2fcf9","datavalue":{"value":{"entity-type":"item","numeric-id":2955913,"id":"Q2955913"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"5c7c2a267d8c46b33f6182520fa8c69b8cdf2578","datavalue":{"value":{"amount":"+0.93859875","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$5BBAC5DE-DCB2-44BC-964E-38769D152C85","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"3ee0299d924a2af692a59e24c9ebba1b893a995f","datavalue":{"value":{"entity-type":"item","numeric-id":4593601,"id":"Q4593601"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"ef2a7f27ba36452c12d8b4fdc6ed3e0961fdab93","datavalue":{"value":{"amount":"+0.93305033","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$1DFA3D4F-95DA-42CD-BF13-A7B3692FA141","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"ae103eba20089b839a20b22937dc0410a9b74810","datavalue":{"value":{"entity-type":"item","numeric-id":5388015,"id":"Q5388015"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"c234d6515cffe84f4bc045067004c1288fb041cb","datavalue":{"value":{"amount":"+0.90654474","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$9F06E3C4-1694-4AEF-A7F1-5F3F0770B0E1","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"007a072acff9fdab4b3c0fa41bde68bb1599e571","datavalue":{"value":{"entity-type":"item","numeric-id":3165917,"id":"Q3165917"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"d27eb83361d42c2b98b142fbc2a783f2afb0a0d5","datavalue":{"value":{"amount":"+0.9015029","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$ACEA9916-A93C-4D22-A00D-64A70803C0E2","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"d7f29fe6e255d47d519409428ddcea50d42e794e","datavalue":{"value":{"entity-type":"item","numeric-id":4888206,"id":"Q4888206"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"9c5d02ab5eb9c8b040697fe7c2f52fbb344c5341","datavalue":{"value":{"amount":"+0.8966035","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$FF382B07-8FCD-48B9-B5E9-CD8E1D1473BC","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"410f29a8b2b2922c261850781adef6b5e2db5b1d","datavalue":{"value":{"entity-type":"item","numeric-id":408405,"id":"Q408405"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"db009116ae6e52da2adffef705efbd41fd2e94bd","datavalue":{"value":{"amount":"+0.8941864","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$65443A9F-9C27-44D0-A270-EFC0B0AB59B7","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"3e8330d756a543936cdb617364010c57503e0e27","datavalue":{"value":{"entity-type":"item","numeric-id":2337423,"id":"Q2337423"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"f93e58f5ec6b85cdff682e0a91570b30e9736ad3","datavalue":{"value":{"amount":"+0.89291817","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$C6B24A3A-5E25-499F-9840-0F9F56F9C50C","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"6f59236eb62fe47ca5af206e59ac54efb30a48a6","datavalue":{"value":{"entity-type":"item","numeric-id":5271025,"id":"Q5271025"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"e68011a9dbd7d297dd6d56dce393bc51d1336d71","datavalue":{"value":{"amount":"+0.89129","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$B38A7F14-EDC5-4DEA-8F61-61514F3815F8","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P1643","hash":"801cfe7fe350f488639d71817416c5ac9de088e0","datavalue":{"value":{"entity-type":"item","numeric-id":5153610,"id":"Q5153610"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P1659":[{"snaktype":"value","property":"P1659","hash":"5401b6f574c16061f7b672b18053e9017d2f8ab7","datavalue":{"value":{"amount":"+0.8909234","unit":"1"},"type":"quantity"},"datatype":"quantity"}],"P1660":[{"snaktype":"value","property":"P1660","hash":"ac3c626774dcd0d16f89557f66586245841a01db","datavalue":{"value":{"entity-type":"item","numeric-id":6767936,"id":"Q6767936"},"type":"wikibase-entityid"},"datatype":"wikibase-item"}]},"qualifiers-order":["P1659","P1660"],"id":"Q584085$383F2AF5-BC48-4CF5-9E7F-27B41EF99854","rank":"normal"}]},"sitelinks":{"mardi":{"site":"mardi","title":"Publication:584085","badges":[],"url":"https://portal.mardi4nfdi.de/wiki/Publication:584085"}}}}}