"A general framework for setting up statistical inference problems"?

[rozsasarpi]

I thought that the intention was to restrict the scope to a few selected data generating processes (likelihood functions) that we commonly use because that seems to be needed to be able to convert a model to inputs for various parameter estimation/model selection tools/packages.

To me stan, pymc3, turing.jl are examples of packages that are close to being general in defining Bayesian statistical problems yet they are also careful with calling themselves general.

In what sense probeye is (intends to be) general?

[aklawonn]

The term "general" was intended to reflect the broad range of problems you should be able to describe or set up with this package. However, I don't have a clear view on what is not possible to model, so maybe we should be more careful with this term. With regard of the flexibility the user has to set up his problem (define his own model, the noise model etc.) I would suggest that we could change this sentence (which in the current version reads "A general framework for setting up parameter estimation problems") to "A flexible framework for setting up parameter estimation problems". But I'm also open to any other suggestions.

[rozsasarpi]

With regard of the flexibility the user has to set up his problem (define his own model, the noise model etc.)

If a user created some custom models, would they be automatically converted to something that can be used by various parameter estimation packages such as pyro, emcee, pymc3, etc, or it would be the task of the user? The automatic conversion feels like a very difficult (if possible at all) task.

"A flexible framework for setting up parameter estimation problems". But I'm also open to any other suggestions.

To me the current description can remain (can be dealt with later), I was more curious about the intentions and the vision.

I don't have a clear view on what is not possible to model

If we picked a random example from the stan or Turing.jl tutorials that would probably be not possible to be modelled and solved in probeye and I think probably not even intended to be covered. I guess something as simple as the 8 schools problem would already be outside the current scope.

[joergfunger]

The eight schools problem should be possible to describe with probeye. We have foreseen parameters characterizing the model, parameters characterizing the noise and parameters characterizing the prior of the model parameters (hyperparameters). As such, a one level hierachy could be modelled. As for the above question, I see our goal to restrict ourselves to a limited set of inference problems, but try to be extensible. And in particular provide an interface to describe the problem in a tool independent version, being able to store that and retrive it, call different inference engines such as Stan, pymc3 or pyro.

[rozsasarpi]

The eight schools problem should be possible to describe with probeye.

Sounds great. Could someone put together an example to demonstrate how this could be done?

[aklawonn]

From a brief glance on the 8-schools-problem, I would also say, that this is possible to reflect in probeye. But still, it might be of value for any user to get a clear statement of what is and what isn't possible to cover with this tool. With respect to your question

If a user created some custom models, would they be automatically converted to something that can be used by various parameter estimation packages such as pyro, emcee, pymc3, etc, or it would be the task of the user? The automatic conversion feels like a very difficult (possible at all) task.

I would say, that if the user describes his/her problem in probeye, it would be the goal to provide at least a solver-functionality within probeye, that uses pyro, emcee, pymc3, etc.. I don't know if we can feed back the probeye problem to these engines in a meaningful way (e.g. by conversion), so that they can continue working on the problem from outside probeye within the respective engine scope.

[aklawonn]

Sounds great. Could someone put together an example to demonstrate how this could be done?

I can have a look at it. It might be a nice example to put in a future documentation.