Package: MDP2 3.0.0

Lars Relund Nielsen

MDP2: Markov Decision Processes (MDPs)

Create and optimize (semi) MDPs with discrete time steps and state space. Both hierarchical and ordinary-traditional MDPs can be modeled.

Authors:Lars Relund Nielsen [aut, cre]

MDP2_3.0.0.tar.gz
MDP2_3.0.0.zip(r-4.7)MDP2_3.0.0.zip(r-4.6)MDP2_3.0.0.zip(r-4.5)
MDP2_3.0.0.tgz(r-4.6-x86_64)MDP2_3.0.0.tgz(r-4.6-arm64)MDP2_3.0.0.tgz(r-4.5-x86_64)MDP2_3.0.0.tgz(r-4.5-arm64)
MDP2_3.0.0.tar.gz(r-4.7-arm64)MDP2_3.0.0.tar.gz(r-4.7-x86_64)MDP2_3.0.0.tar.gz(r-4.6-arm64)MDP2_3.0.0.tar.gz(r-4.6-x86_64)
manual.pdf |manual.html✨
DESCRIPTION |NEWS
card.svg |card.png
MDP2/json (API)

# Install 'MDP2' in R:
install.packages('MDP2', repos = c('https://relund.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/relund/mdp/issues

Pkgdown/docs site:https://relund.github.io

Uses libs:
  • openblas– Optimized BLAS
  • c++– GNU Standard C++ Library v3

On CRAN:

Conda:

openblascpp

6.01 score 2 stars 41 scripts 209 downloads 23 exports 24 dependencies

Last updated from:ade64bc931. Checks:12 OK, 1 FAIL. Indexed: yes.

TargetResultTimeFilesSyslog
linux-devel-arm64OK163
linux-devel-x86_64OK174
source / vignettesOK250
linux-release-arm64OK171
linux-release-x86_64OK185
macos-release-arm64OK182
macos-release-x86_64OK384
macos-oldrel-arm64OK144
macos-oldrel-x86_64OK346
windows-develOK172
windows-releaseOK113
windows-oldrelOK158
wasm-releaseFAIL123

Exports:binary_action_writerbinary_mdp_writerconvert_binary_to_hmpconvert_hmp_to_binaryget_bin_info_actionsget_bin_info_statesget_hypergraphget_infoget_policyget_rpoget_steady_state_prget_w_idxhmp_mdp_writerload_mdpmemory_mdp_writerplot_hypergraphrandom_hmdprun_calc_weightsrun_policy_ite_averun_policy_ite_discountrun_value_itesave_mdpset_policy

Dependencies:clicpp11diagramdplyrgenericsgluelifecyclemagrittrpillarpkgconfigpurrrR6RcppRcppArmadillorlangshapestringistringrtibbletidyrtidyselectutf8vctrswithr

Building an MDP model
An infinite semi-MDP | A finite-horizon semi-MDP | An infinite-horizon HMDP | References
Rendered frombuilding.Rmdusingknitr::rmarkdown

Last update: 2026-06-04
Started: 2023-01-27

Solving a finite-horizon semi-MDP
A finite-horizon semi-MDP | Example | References
Rendered fromfinite-mdp.Rmdusingknitr::rmarkdown

Last update: 2026-06-04
Started: 2023-01-27

An infinite-horizon HMDP
Example | References
Rendered frominfinite-hmdp.Rmdusingknitr::rmarkdown

Last update: 2026-06-04
Started: 2023-01-27

Solving an infinite-horizon semi-MDP
An infinite-horizon semi-MDP | Example | References
Rendered frominfinite-mdp.Rmdusingknitr::rmarkdown

Last update: 2026-06-04
Started: 2023-01-27

The MDP2 package
References
Rendered frommdp2.Rmdusingknitr::rmarkdown

Last update: 2026-06-04
Started: 2022-12-16

Readme and manuals

Help Manual

Help pageTopics
Function for writing actions of a HMDP model to binary files. The function defines sub-functions which can be used to define actions saved in a set of binary files. It is assumed that the states have been defined using 'binary_mdp_writer' and that the id of the states is known (can be retrieved using e.g. 'state_idx_df').binary_action_writer
Function for writing an HMDP model to binary files. The function defines sub-functions which can be used to define an HMDP model saved in a set of binary files.binary_mdp_writer
Convert a HMDP model stored in binary format to a 'hmp' (XML) file. The function simply parse the binary files and create 'hmp' files using the 'hmp_mdp_writer()'.convert_binary_to_hmp
Convert a HMDP model stored in a hmp (xml) file to binary file format.convert_hmp_to_binary
Info about the actions in the HMDP model under consideration.get_bin_info_actions
Info about the states in the binary files of the HMDP model under consideration.get_bin_info_states
Return the (parts of) state-expanded hypergraphget_hypergraph
Information about the MDPget_info
Get parts of the optimal policy.get_policy
Calculate the retention pay-off (RPO) or opportunity cost for some states.get_rpo
Calculate the steady state transition probabilities for the founder process (level 0).get_steady_state_pr
Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.get_w_idx
Function for writing an HMDP model to a hmp file (XML). The function define sub-functions which can be used to define an HMDP model stored in a hmp file.hmp_mdp_writer
Load the HMDP model defined in the binary files. The model are created in memory using the external C++ library.load_mdp
Function for building an HMDP model directly in memory.memory_mdp_writer
Plot parts of the state expanded hypergraph.plot_hypergraph
Plot the state-expanded hypergraph of the MDP.plot.HMDP
Generate a "random" HMDP stored in a set of binary files.random_hmdp
Calculate weights based on current policy. Normally run after an optimal policy has been found.run_calc_weights
Perform policy iteration using the average expected-weight Bellman operator on the MDP.run_policy_ite_ave
Perform policy iteration using the discounted expected-weight Bellman operator on the MDP.run_policy_ite_discount
Perform value iteration on the MDP.run_value_ite
Save the MDP to binary filessave_mdp
Modify the current policy by setting policy action of states.set_policy