Comments (2)
After thinking twice, I think it makes more sense provide a QuantityManager
. This quantity manager can then be forwarded to independent reward components satisfying some callable protocole (which means it would be a lambda, a function or a class defining __class__
aka functor). It would more modular and easier to extend this way. To be computationally efficient, this quantity manager should heavily rely on caching. This cache must be clear manually before computing any reward component, then quantities would be computed only the first time it is requested, or never if not used. Here is a snipped:
class QuantityManager:
def __init__(self, robot: BaseJiminyRobot, quantities: Dict[str, Callable[[], Any]]) -> None:
self.robot = env.robot
self.quantities = quantities
self._cache : Dict[str, Any]
def __getattr__(self, name: str) -> Any:
return self.__cache.setdefault(name, self.quantities.get(name))
def __item__(self, name:str) -> Any:
return getattr(self, name)
def reset() -> None:
self.__cache.clear()
class RelativePose:
def __init__(self, robot: BaseJiminyRobot, first_name: str, second_name: str) -> None:
first_index = robot.pinocchio_model.getFrameId(first_name)
second_index = robot.pinocchio_model.getFrameId(second_name)
self.first_pose = self.robot.oMf[first_index]
self.second_pose = self.robot.oMf[second_index]
def __call__(self) -> pin.SE3:
return self.first_pose.actInv(self.second_pose)
def foot_placement_reward(quantities: QuantityManager) -> float:
return np.linalg.norm(quantities.foot_pose_rel.translation)
[...]
foot_pose_rel_qty = RelativePose(env.robot, "LeftSole", "RightSole")
quantities = QuantityManager(env.robot, {"foot_pose_rel": foot_pose_rel_qty}))
There would be a reward manager, taking a quantity manager and a set of reward components as input. It would expose a single compute_reward
method, that would first call reset
on the quantity manager, and then all reward components individually. Eventually, it would also feature a reset
method that would call the reset
method of each reward components if any. It may be beneficial to keep termination conditions and reward evaluation together, to avoid computing quantities twice. If so, then it would be more tricky to determine when to reset cached quantities automatically.
from jiminy.
This issue has been addressed. Closing. #784 #786 #787 #792
from jiminy.
Related Issues (20)
- [core] Wrong operator in `AbstractPerlinProcess::getWavelength`
- [gym_jiminy/common] More generic `sample` utility.
- [core] Get contact frame index once for all at reset.
- [core] Wrong force sensor in presence of external forces
- [gym_jiminy/toolbox] `quat_to_rpy` is numerically unstable.
- [core] Broken flexible model generation. HOT 1
- [core/python] Do not use plain C `typeid`.
- [python] Use `weakref.proxy` instead of `weakref.ref` to simplify logic
- [misc] Easy install instructions are not working.
- [misc] Building from source does not support `Pyenv`
- [core] Fix infinite loop !
- [core] Restore constraint lambda in case of integration failure. HOT 3
- [core] Move controller and sensor update periods at robot-level.
- [core] Move set /get engine options at C++ level. HOT 1
- `build_robot_from_log` no longer works due to backlash HOT 5
- [gym_jiminy] BaseJiminyEnv doesn't support single-robot simulators with a robot name different from "" HOT 1
- [gym] Fix typing and docs RLlib utilities with ray >= 2.9
- Constraint solver leads to non-consistent integration HOT 2
- [python/gym] Unable to run simple Atlas simulation because truncated is true immediately HOT 9
- [gym][wrappers] StackObservation is not preserving the order for OrderedDict observation spaces. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jiminy.