Comments (7)
Thank you for the kind words!
To get it to work on a new environment you should:
- Create a new environment. For this, you can take inspiration from the
Navigation2D
environment in this repo. This is a standard gym environment (withseed
,reset
, andstep
functions), but also requires two extra functions:sample_tasks(num_tasks)
which samplesnum_tasks
tasks, andreset_task(task)
which sets the task according totask
.sample_tasks
returns a list of tasks (dictionaries with task specifications), andreset_task
takes a dictionary (task specification) as input. - Register the environment here, to be able to call it with
gym.make()
. - Create a configuration file. You can take inspiration from the
2d-navigation.yaml
configuration file. The most important thing in the config file is theenv-name
(which corresponds to the name you gave your environment when you registered it at the previous step), andenv-kwargs
if you have extra arguments to pass your environment (if you don't have any, remove theenv-kwargs
entry).
And you should be able to call train.py
on your new configuration file:
python train.py --config new-config.yaml --output-folder new-environment --seed 1 --num-workers 8
The script should detect if your problem has discrete or continuous actions (from action_space
in the gym environment you defined), and chose either a Normal or Categorical policy accordingly.
Let me know if that worked for you!
from pytorch-maml-rl.
I'm sorry for reply you so late.
The full error of my code is
File "/home/rao/桌面/pytorch-maml-rl-master (1)/pytorch-maml-rl-master/maml_rl/samplers/multi_task_sampler.py", line 297, in create_episodes for item in self.sample_trajectories(params=params): File "/home/rao/桌面/pytorch-maml-rl-master (1)/pytorch-maml-rl-master/maml_rl/samplers/multi_task_sampler.py", line 308, in sample_trajectories observations = self.envs.reset() File "/home/rao/anaconda3/lib/python3.7/site-packages/gym/vector/vector_env.py", line 60, in reset return self.reset_wait() File "/home/rao/anaconda3/lib/python3.7/site-packages/gym/vector/sync_vector_env.py", line 64, in reset_wait observation = env.reset() File "/home/rao/下载/gym-carla-master/gym_carla/envs/carla_env.py", line 189, in reset self._clear_all_actors(['sensor.other.collision', 'sensor.lidar.ray_cast', 'sensor.camera.rgb', 'vehicle.*', 'controller.ai.walker', 'walker.*']) File "/home/rao/下载/gym-carla-master/gym_carla/envs/carla_env.py", line 700, in _clear_all_actors for actor in self.world.get_actors().filter(actor_filter): RuntimeError: time-out of 50000ms while waiting for the simulator, make sure the simulator is ready and connected to localhost:2000 It should be noted that my code is try to collected a simlator which is successful in the __init__, but run time error in the reset part. And here is my total env code
from future import division
import copy
import numpy as np
import pygame
import random
import time
from skimage.transform import resize
import gym
from gym import spaces
from gym.utils import seeding
import carla
from gym_carla.envs.render import BirdeyeRender
from gym_carla.envs.route_planner import RoutePlanner
from gym_carla.envs.misc import *
class CarlaEnv(gym.Env):
"""An OpenAI gym wrapper for CARLA simulator."""
def init(self, params,low=2.0,high=10.0, task={}):
# parameters
self.display_size = params['display_size'] # rendering screen size
self.max_past_step = params['max_past_step']
self.number_of_vehicles = params['number_of_vehicles']
self.number_of_walkers = params['number_of_walkers']
self.dt = params['dt']
self.task_mode = params['task_mode']
self.max_time_episode = params['max_time_episode']
self.max_waypt = params['max_waypt']
self.obs_range = params['obs_range']
self.lidar_bin = params['lidar_bin']
self.d_behind = params['d_behind']
self.obs_size = int(self.obs_range/self.lidar_bin)
self.out_lane_thres = params['out_lane_thres']
# self.desired_speed = params['desired_speed']
self.max_ego_spawn_times = params['max_ego_spawn_times']
self.display_route = params['display_route']
self._task=task
self._goal=task.get('goal',np.zeros(1,dtype=np.float32))
self.low= low
self.high=high
# self._state=np.zeros(2,dtype=np.float32)
if 'pixor' in params.keys():
self.pixor = params['pixor']
self.pixor_size = params['pixor_size']
else:
self.pixor = False
# Destination
if params['task_mode'] == 'roundabout':
self.dests = [[4.46, -61.46, 0], [-49.53, -2.89, 0], [-6.48, 55.47, 0], [35.96, 3.33, 0]]
else:
self.dests = None
# action and observation spaces
self.discrete = params['discrete']
self.discrete_act = [params['discrete_acc'], params['discrete_steer']] # acc, steer
self.n_acc = len(self.discrete_act[0])
self.n_steer = len(self.discrete_act[1])
if self.discrete:
self.action_space = spaces.Discrete(self.n_acc*self.n_steer)
else:
self.action_space = spaces.Box(np.array([params['continuous_accel_range'][0],
params['continuous_steer_range'][0]]), np.array([params['continuous_accel_range'][1],
params['continuous_steer_range'][1]]), dtype=np.float32) # acc, steer
observation_space_dict = {
'camera': spaces.Box(low=0, high=255, shape=(self.obs_size, self.obs_size, 3), dtype=np.uint8),
'lidar': spaces.Box(low=0, high=255, shape=(self.obs_size, self.obs_size, 3), dtype=np.uint8),
'birdeye': spaces.Box(low=0, high=255, shape=(self.obs_size, self.obs_size, 3), dtype=np.uint8),
'state': spaces.Box(np.array([-2, -1, -5, 0]), np.array([2, 1, 30, 1]), dtype=np.float32)
}
if self.pixor:
observation_space_dict.update({
'roadmap': spaces.Box(low=0, high=255, shape=(self.obs_size, self.obs_size, 3), dtype=np.uint8),
'vh_clas': spaces.Box(low=0, high=1, shape=(self.pixor_size, self.pixor_size, 1), dtype=np.float32),
'vh_regr': spaces.Box(low=-5, high=5, shape=(self.pixor_size, self.pixor_size, 6), dtype=np.float32),
'pixor_state': spaces.Box(np.array([-1000, -1000, -1, -1, -5]), np.array([1000, 1000, 1, 1, 20]), dtype=np.float32)
})
self.observation_space = spaces.Dict(observation_space_dict)
# Connect to carla server and get world object
print('connecting to Carla server...')
client = carla.Client('localhost', params['port'])
client.set_timeout(50.0)
self.world = client.load_world(params['town'])
# self.spectator = self.world.get_spectator()
# self.blueprint_library = self.world.get_blueprint_library()
print('Carla server connected!')
# Set weather
self.world.set_weather(carla.WeatherParameters.ClearNoon)
# Get spawn points
self.vehicle_spawn_points = list(self.world.get_map().get_spawn_points())
self.walker_spawn_points = []
for i in range(self.number_of_walkers):
spawn_point = carla.Transform()
loc = self.world.get_random_location_from_navigation()
if (loc != None):
spawn_point.location = loc
self.walker_spawn_points.append(spawn_point)
# Create the ego vehicle blueprint
self.ego_bp = self._create_vehicle_bluepprint(params['ego_vehicle_filter'], color='49,8,8')
# Collision sensor
self.collision_hist = [] # The collision history
self.collision_hist_l = 1 # collision history length
self.collision_bp = self.world.get_blueprint_library().find('sensor.other.collision')
# Lidar sensor
self.lidar_data = None
self.lidar_height = 2.1
self.lidar_trans = carla.Transform(carla.Location(x=0.0, z=self.lidar_height))
self.lidar_bp = self.world.get_blueprint_library().find('sensor.lidar.ray_cast')
self.lidar_bp.set_attribute('channels', '32')
self.lidar_bp.set_attribute('range', '5000')
# Camera sensor
self.camera_img = np.zeros((self.obs_size, self.obs_size, 3), dtype=np.uint8)
self.camera_trans = carla.Transform(carla.Location(x=0.8, z=1.7))
self.camera_bp = self.world.get_blueprint_library().find('sensor.camera.rgb')
# Modify the attributes of the blueprint to set image resolution and field of view.
self.camera_bp.set_attribute('image_size_x', str(self.obs_size))
self.camera_bp.set_attribute('image_size_y', str(self.obs_size))
self.camera_bp.set_attribute('fov', '110')
# Set the time in seconds between sensor captures
self.camera_bp.set_attribute('sensor_tick', '0.02')
# Set fixed simulation step for synchronous mode
self.settings = self.world.get_settings()
self.settings.fixed_delta_seconds = self.dt
# Record the time of total steps and resetting steps
self.reset_step = 0
self.total_step = 0
# Initialize the renderer
self._init_renderer()
self.seed()
self.task=task
# Get pixel grid points
if self.pixor:
x, y = np.meshgrid(np.arange(self.pixor_size), np.arange(self.pixor_size)) # make a canvas with coordinates
x, y = x.flatten(), y.flatten()
self.pixel_grid = np.vstack((x, y)).T
def sample_tasks(self, num_tasks):
goals = self.np_random.uniform(self.low, self.high, size=(num_tasks, 1))
tasks = [{'goal': goal} for goal in goals]
return tasks
def reset_task(self, task):
self._task = task
self._goal = task['goal']
def reset(self, env=True):
# Clear sensor objects
self.collision_sensor = None
self.lidar_sensor = None
self.camera_sensor = None
# Delete sensors, vehicles and walkers
self._clear_all_actors(['sensor.other.collision', 'sensor.lidar.ray_cast', 'sensor.camera.rgb', 'vehicle.*', 'controller.ai.walker', 'walker.*'])
# Disable sync mode
self._set_synchronous_mode(False)
# Spawn surrounding vehicles
random.shuffle(self.vehicle_spawn_points)
count = self.number_of_vehicles
if count > 0:
for spawn_point in self.vehicle_spawn_points:
if self._try_spawn_random_vehicle_at(spawn_point, number_of_wheels=[4]):
count -= 1
if count <= 0:
break
while count > 0:
if self._try_spawn_random_vehicle_at(random.choice(self.vehicle_spawn_points), number_of_wheels=[4]):
count -= 1
# Spawn pedestrians
random.shuffle(self.walker_spawn_points)
count = self.number_of_walkers
if count > 0:
for spawn_point in self.walker_spawn_points:
if self._try_spawn_random_walker_at(spawn_point):
count -= 1
if count <= 0:
break
while count > 0:
if self._try_spawn_random_walker_at(random.choice(self.walker_spawn_points)):
count -= 1
# Get actors polygon list
self.vehicle_polygons = []
vehicle_poly_dict = self._get_actor_polygons('vehicle.*')
self.vehicle_polygons.append(vehicle_poly_dict)
self.walker_polygons = []
walker_poly_dict = self._get_actor_polygons('walker.*')
self.walker_polygons.append(walker_poly_dict)
# Spawn the ego vehicle
ego_spawn_times = 0
while True:
if ego_spawn_times > self.max_ego_spawn_times:
self.reset()
if self.task_mode == 'random':
transform = random.choice(self.vehicle_spawn_points)
if self.task_mode == 'roundabout':
self.start=[52.1+np.random.uniform(-5,5),-4.2, 178.66] # random
# self.start=[52.1,-4.2, 178.66] # static
transform = set_carla_transform(self.start)
if self._try_spawn_ego_vehicle_at(transform):
break
else:
ego_spawn_times += 1
time.sleep(0.1)
# Add collision sensor
self.collision_sensor = self.world.spawn_actor(self.collision_bp, carla.Transform(), attach_to=self.ego)
self.collision_sensor.listen(lambda event: get_collision_hist(event))
# self.spectator.set_transform(self.get_viewer_location())
def get_collision_hist(event):
impulse = event.normal_impulse
intensity = np.sqrt(impulse.x**2 + impulse.y**2 + impulse.z**2)
self.collision_hist.append(intensity)
if len(self.collision_hist)>self.collision_hist_l:
self.collision_hist.pop(0)
self.collision_hist = []
# Add lidar sensor
self.lidar_sensor = self.world.spawn_actor(self.lidar_bp, self.lidar_trans, attach_to=self.ego)
self.lidar_sensor.listen(lambda data: get_lidar_data(data))
def get_lidar_data(data):
self.lidar_data = data
# Add camera sensor
self.camera_sensor = self.world.spawn_actor(self.camera_bp, self.camera_trans, attach_to=self.ego)
self.camera_sensor.listen(lambda data: get_camera_img(data))
def get_camera_img(data):
array = np.frombuffer(data.raw_data, dtype = np.dtype("uint8"))
array = np.reshape(array, (data.height, data.width, 4))
array = array[:, :, :3]
array = array[:, :, ::-1]
self.camera_img = array
# Update timesteps
self.time_step=0
self.reset_step+=1
# Enable sync mode
self.settings.synchronous_mode = True
self.world.apply_settings(self.settings)
self.routeplanner = RoutePlanner(self.ego, self.max_waypt)
self.waypoints, _, self.vehicle_front = self.routeplanner.run_step()
# Set ego information for render
self.birdeye_render.set_hero(self.ego, self.ego.id)
return self._get_obs()
def step(self, action):
# Calculate acceleration and steering
if self.discrete:
acc = self.discrete_act[0][action//self.n_steer]
steer = self.discrete_act[1][action%self.n_steer]
else:
acc = action[0]
steer = action[1]
# Convert acceleration to throttle and brake
if acc > 0:
throttle = np.clip(acc/3,0,1)
brake = 0
else:
throttle = 0
brake = np.clip(-acc/8,0,1)
# Apply control
act = carla.VehicleControl(throttle=float(throttle), steer=float(-steer), brake=float(brake))
self.ego.apply_control(act)
self.desired_speed=self._goal
self.world.tick()
# Append actors polygon list
vehicle_poly_dict = self._get_actor_polygons('vehicle.*')
self.vehicle_polygons.append(vehicle_poly_dict)
while len(self.vehicle_polygons) > self.max_past_step:
self.vehicle_polygons.pop(0)
walker_poly_dict = self._get_actor_polygons('walker.*')
self.walker_polygons.append(walker_poly_dict)
while len(self.walker_polygons) > self.max_past_step:
self.walker_polygons.pop(0)
# route planner
self.waypoints, _, self.vehicle_front = self.routeplanner.run_step()
# state information
info = {
'waypoints': self.waypoints,
'vehicle_front': self.vehicle_front
}
# Update timesteps
self.time_step += 1
self.total_step += 1
return (self._get_obs(), self._get_reward(), self._terminal(), copy.deepcopy(info),{'task': self._task})
def seed(self, seed=None):
self.np_random, seed = seeding.np_random(seed)
return [seed]
def render(self, mode):
pass
def _create_vehicle_bluepprint(self, actor_filter, color=None, number_of_wheels=[4]):
"""Create the blueprint for a specific actor type.
Args:
actor_filter: a string indicating the actor type, e.g, 'vehicle.lincoln*'.
Returns:
bp: the blueprint object of carla.
"""
blueprints = self.world.get_blueprint_library().filter(actor_filter)
blueprint_library = []
for nw in number_of_wheels:
blueprint_library = blueprint_library + [x for x in blueprints if int(x.get_attribute('number_of_wheels')) == nw]
bp = random.choice(blueprint_library)
if bp.has_attribute('color'):
if not color:
color = random.choice(bp.get_attribute('color').recommended_values)
bp.set_attribute('color', color)
return bp
def _init_renderer(self):
"""Initialize the birdeye view renderer.
"""
pygame.init()
self.display = pygame.display.set_mode(
(self.display_size * 3, self.display_size),
pygame.HWSURFACE | pygame.DOUBLEBUF)
pixels_per_meter = self.display_size / self.obs_range
pixels_ahead_vehicle = (self.obs_range/2 - self.d_behind) * pixels_per_meter
birdeye_params = {
'screen_size': [self.display_size, self.display_size],
'pixels_per_meter': pixels_per_meter,
'pixels_ahead_vehicle': pixels_ahead_vehicle
}
self.birdeye_render = BirdeyeRender(self.world, birdeye_params)
def _set_synchronous_mode(self, synchronous = True):
"""Set whether to use the synchronous mode.
"""
self.settings.synchronous_mode = synchronous
self.world.apply_settings(self.settings)
def _try_spawn_random_vehicle_at(self, transform, number_of_wheels=[4]):
"""Try to spawn a surrounding vehicle at specific transform with random bluprint.
Args:
transform: the carla transform object.
Returns:
Bool indicating whether the spawn is successful.
"""
blueprint = self._create_vehicle_bluepprint('vehicle.*', number_of_wheels=number_of_wheels)
blueprint.set_attribute('role_name', 'autopilot')
vehicle = self.world.try_spawn_actor(blueprint, transform)
if vehicle is not None:
vehicle.set_autopilot()
return True
return False
def _try_spawn_random_walker_at(self, transform):
"""Try to spawn a walker at specific transform with random bluprint.
Args:
transform: the carla transform object.
Returns:
Bool indicating whether the spawn is successful.
"""
walker_bp = random.choice(self.world.get_blueprint_library().filter('walker.*'))
# set as not invencible
if walker_bp.has_attribute('is_invincible'):
walker_bp.set_attribute('is_invincible', 'false')
walker_actor = self.world.try_spawn_actor(walker_bp, transform)
if walker_actor is not None:
walker_controller_bp = self.world.get_blueprint_library().find('controller.ai.walker')
walker_controller_actor = self.world.spawn_actor(walker_controller_bp, carla.Transform(), walker_actor)
# start walker
walker_controller_actor.start()
# set walk to random point
walker_controller_actor.go_to_location(self.world.get_random_location_from_navigation())
# random max speed
walker_controller_actor.set_max_speed(1 + random.random()) # max speed between 1 and 2 (default is 1.4 m/s)
return True
return False
def _try_spawn_ego_vehicle_at(self, transform):
"""Try to spawn the ego vehicle at specific transform.
Args:
transform: the carla transform object.
Returns:
Bool indicating whether the spawn is successful.
"""
vehicle = None
# Check if ego position overlaps with surrounding vehicles
overlap = False
for idx, poly in self.vehicle_polygons[-1].items():
poly_center = np.mean(poly, axis=0)
ego_center = np.array([transform.location.x, transform.location.y])
dis = np.linalg.norm(poly_center - ego_center)
if dis > 8:
continue
else:
overlap = True
break
if not overlap:
vehicle = self.world.try_spawn_actor(self.ego_bp, transform)
if vehicle is not None:
self.ego=vehicle
return True
return False
def _get_actor_polygons(self, filt):
"""Get the bounding box polygon of actors.
Args:
filt: the filter indicating what type of actors we'll look at.
Returns:
actor_poly_dict: a dictionary containing the bounding boxes of specific actors.
"""
actor_poly_dict={}
for actor in self.world.get_actors().filter(filt):
# Get x, y and yaw of the actor
trans=actor.get_transform()
x=trans.location.x
y=trans.location.y
yaw=trans.rotation.yaw/180*np.pi
# Get length and width
bb=actor.bounding_box
l=bb.extent.x
w=bb.extent.y
# Get bounding box polygon in the actor's local coordinate
poly_local=np.array([[l,w],[l,-w],[-l,-w],[-l,w]]).transpose()
# Get rotation matrix to transform to global coordinate
R=np.array([[np.cos(yaw),-np.sin(yaw)],[np.sin(yaw),np.cos(yaw)]])
# Get global bounding box polygon
poly=np.matmul(R,poly_local).transpose()+np.repeat([[x,y]],4,axis=0)
actor_poly_dict[actor.id]=poly
return actor_poly_dict
def _get_obs(self):
"""Get the observations."""
## Birdeye rendering
self.birdeye_render.vehicle_polygons = self.vehicle_polygons
self.birdeye_render.walker_polygons = self.walker_polygons
self.birdeye_render.waypoints = self.waypoints
# birdeye view with roadmap and actors
birdeye_render_types = ['roadmap', 'actors']
if self.display_route:
birdeye_render_types.append('waypoints')
self.birdeye_render.render(self.display, birdeye_render_types)
birdeye = pygame.surfarray.array3d(self.display)
birdeye = birdeye[0:self.display_size, :, :]
birdeye = display_to_rgb(birdeye, self.obs_size)
# Roadmap
if self.pixor:
roadmap_render_types = ['roadmap']
if self.display_route:
roadmap_render_types.append('waypoints')
self.birdeye_render.render(self.display, roadmap_render_types)
roadmap = pygame.surfarray.array3d(self.display)
roadmap = roadmap[0:self.display_size, :, :]
roadmap = display_to_rgb(roadmap, self.obs_size)
# Add ego vehicle
for i in range(self.obs_size):
for j in range(self.obs_size):
if abs(birdeye[i, j, 0] - 255)<20 and abs(birdeye[i, j, 1] - 0)<20 and abs(birdeye[i, j, 0] - 255)<20:
roadmap[i, j, :] = birdeye[i, j, :]
# Display birdeye image
birdeye_surface = rgb_to_display_surface(birdeye, self.display_size)
self.display.blit(birdeye_surface, (0, 0))
## Lidar image generation
point_cloud = []
# Get point cloud data
for location in self.lidar_data:
point_cloud.append([location.x, location.y, -location.z])
point_cloud = np.array(point_cloud)
# Separate the 3D space to bins for point cloud, x and y is set according to self.lidar_bin,
# and z is set to be two bins.
y_bins = np.arange(-(self.obs_range - self.d_behind), self.d_behind+self.lidar_bin, self.lidar_bin)
x_bins = np.arange(-self.obs_range/2, self.obs_range/2+self.lidar_bin, self.lidar_bin)
z_bins = [-self.lidar_height-1, -self.lidar_height+0.25, 1]
# Get lidar image according to the bins
lidar, _ = np.histogramdd(point_cloud, bins=(x_bins, y_bins, z_bins))
lidar[:,:,0] = np.array(lidar[:,:,0]>0, dtype=np.uint8)
lidar[:,:,1] = np.array(lidar[:,:,1]>0, dtype=np.uint8)
# Add the waypoints to lidar image
if self.display_route:
wayptimg = (birdeye[:,:,0] <= 10) * (birdeye[:,:,1] <= 10) * (birdeye[:,:,2] >= 240)
else:
wayptimg = birdeye[:,:,0] < 0 # Equal to a zero matrix
wayptimg = np.expand_dims(wayptimg, axis=2)
wayptimg = np.fliplr(np.rot90(wayptimg, 3))
# Get the final lidar image
lidar = np.concatenate((lidar, wayptimg), axis=2)
lidar = np.flip(lidar, axis=1)
lidar = np.rot90(lidar, 1)
lidar = lidar * 255
# Display lidar image
lidar_surface = rgb_to_display_surface(lidar, self.display_size)
self.display.blit(lidar_surface, (self.display_size, 0))
## Display camera image
camera = resize(self.camera_img, (self.obs_size, self.obs_size)) * 255
camera_surface = rgb_to_display_surface(camera, self.display_size)
self.display.blit(camera_surface, (self.display_size * 2, 0))
# Display on pygame
pygame.display.flip()
# State observation
ego_trans = self.ego.get_transform()
ego_x = ego_trans.location.x
ego_y = ego_trans.location.y
ego_yaw = ego_trans.rotation.yaw/180*np.pi
lateral_dis, w = get_preview_lane_dis(self.waypoints, ego_x, ego_y)
delta_yaw = np.arcsin(np.cross(w,
np.array(np.array([np.cos(ego_yaw), np.sin(ego_yaw)]))))
v = self.ego.get_velocity()
speed = np.sqrt(v.x**2 + v.y**2)
state = np.array([lateral_dis, - delta_yaw, speed, self.vehicle_front])
if self.pixor:
## Vehicle classification and regression maps (requires further normalization)
vh_clas = np.zeros((self.pixor_size, self.pixor_size))
vh_regr = np.zeros((self.pixor_size, self.pixor_size, 6))
# Generate the PIXOR image. Note in CARLA it is using left-hand coordinate
# Get the 6-dim geom parametrization in PIXOR, here we use pixel coordinate
for actor in self.world.get_actors().filter('vehicle.*'):
x, y, yaw, l, w = get_info(actor)
x_local, y_local, yaw_local = get_local_pose((x, y, yaw), (ego_x, ego_y, ego_yaw))
if actor.id != self.ego.id:
if abs(y_local)<self.obs_range/2+1 and x_local<self.obs_range-self.d_behind+1 and x_local>-self.d_behind-1:
x_pixel, y_pixel, yaw_pixel, l_pixel, w_pixel = get_pixel_info(
local_info=(x_local, y_local, yaw_local, l, w),
d_behind=self.d_behind, obs_range=self.obs_range, image_size=self.pixor_size)
cos_t = np.cos(yaw_pixel)
sin_t = np.sin(yaw_pixel)
logw = np.log(w_pixel)
logl = np.log(l_pixel)
pixels = get_pixels_inside_vehicle(
pixel_info=(x_pixel, y_pixel, yaw_pixel, l_pixel, w_pixel),
pixel_grid=self.pixel_grid)
for pixel in pixels:
vh_clas[pixel[0], pixel[1]] = 1
dx = x_pixel - pixel[0]
dy = y_pixel - pixel[1]
vh_regr[pixel[0], pixel[1], :] = np.array(
[cos_t, sin_t, dx, dy, logw, logl])
# Flip the image matrix so that the origin is at the left-bottom
vh_clas = np.flip(vh_clas, axis=0)
vh_regr = np.flip(vh_regr, axis=0)
# Pixor state, [x, y, cos(yaw), sin(yaw), speed]
pixor_state = [ego_x, ego_y, np.cos(ego_yaw), np.sin(ego_yaw), speed]
obs = {
'camera':camera.astype(np.uint8),
'lidar':lidar.astype(np.uint8),
'birdeye':birdeye.astype(np.uint8),
'state': state,
}
if self.pixor:
obs.update({
'roadmap':roadmap.astype(np.uint8),
'vh_clas':np.expand_dims(vh_clas, -1).astype(np.float32),
'vh_regr':vh_regr.astype(np.float32),
'pixor_state': pixor_state,
})
return obs
def _get_reward(self):
"""Calculate the step reward."""
# reward for speed tracking
v = self.ego.get_velocity()
speed = np.sqrt(v.x2 + v.y2)
r_speed = -abs(speed - self.desired_speed)
# reward for collision
r_collision = 0
if len(self.collision_hist) > 0:
r_collision = -1
# reward for steering:
r_steer = -self.ego.get_control().steer**2
# reward for out of lane
ego_x, ego_y = get_pos(self.ego)
dis, w = get_lane_dis(self.waypoints, ego_x, ego_y)
r_out = 0
if abs(dis) > self.out_lane_thres:
r_out = -1
# longitudinal speed
lspeed = np.array([v.x, v.y])
lspeed_lon = np.dot(lspeed, w)
# cost for too fast
r_fast = 0
if lspeed_lon > self.desired_speed:
r_fast = -1
# cost for lateral acceleration
r_lat = - abs(self.ego.get_control().steer) * lspeed_lon**2
r = 200*r_collision + 1*lspeed_lon + 10*r_fast + 1*r_out + r_steer*5 + 0.2*r_lat - 0.1
return r
def _terminal(self):
"""Calculate whether to terminate the current episode."""
# Get ego state
ego_x, ego_y = get_pos(self.ego)
# If collides
if len(self.collision_hist)>0:
return True
# If reach maximum timestep
if self.time_step>self.max_time_episode:
return True
# If at destination
if self.dests is not None: # If at destination
for dest in self.dests:
if np.sqrt((ego_x-dest[0])**2+(ego_y-dest[1])**2)<4:
return True
# If out of lane
dis, _ = get_lane_dis(self.waypoints, ego_x, ego_y)
if abs(dis) > self.out_lane_thres:
return True
return False
def _clear_all_actors(self, actor_filters):
"""Clear specific actors."""
for actor_filter in actor_filters:
for actor in self.world.get_actors().filter(actor_filter):
if actor.is_alive:
if actor.type_id == 'controller.ai.walker':
actor.stop()
actor.destroy()
`
Thanks for your reply.
from pytorch-maml-rl.
Thanks for your reply! I will try it!
from pytorch-maml-rl.
Thanks for your reply! I have some questions about the reset_Wait. When i reset the env, there are alwanys read time eror.
from pytorch-maml-rl.
Can you give me some reason, thanks a lot
from pytorch-maml-rl.
Can you give the full traceback of the error? There is no reset_wait
function in this repo, do you mean the reset_task
function?
It would also be helpful if you could attach the code of your environment here as well.
from pytorch-maml-rl.
I don't know exactly what the problem can be since I'm not familiar with CARLA, but for these kinds of environments it could be an incompatibility with multiprocessing. I imagine your environment works otherwise (you can initialize it, and call reset
/step
), so one thing you could try is to use it with a VectorEnv from gym, which is also using multiprocessing, and see if you get a similar error.
from pytorch-maml-rl.
Related Issues (20)
- AttributeError: Can't pickle local object 'make_env.<locals>._make_env' HOT 4
- Cuda Support Issue HOT 2
- what is the mean of train_episodes and valid_episodes? HOT 4
- If I want to use the the meta-parameters to adapt to new task, what should I do? HOT 2
- TypeError: list indices must be integers or slices, not str HOT 1
- Fails to converge on bandit tasks HOT 1
- Can not read your env in the Jupyter HOT 1
- how can I adapt maml on my own environment? HOT 1
- How do you get the baseline curve in Fig5 in your paper? HOT 1
- train_returns and valid_returns seems to be equal
- where to setup render value of mujoco environment? HOT 1
- is it posible to combine DPPG with MAML?
- How to solve the problem of Nan value during training?
- The progress bar doesn't increase at all HOT 4
- raise EOFError
- EOFError BrokenPipeError: [WinError 109]
- Traceback (most recent call last): File "D:\Anaconda3\envs\testenv\lib\multiprocessing\process.py", line 258, in _bootstrap self.run() File "F:\aa_maml\MAML-Pytorch-RL-master\maml_rl\envs\subproc_vec_env.py", line 61, in run command, data = self.remote.recv() File "D:\Anaconda3\envs\testenv\lib\multiprocessing\connection.py", line 312, in _recv_bytes nread, err = ov.GetOverlappedResult(True) File "D:\Anaconda3\envs\testenv\lib\multiprocessing\connection.py", line 250, in recv buf = self._recv_bytes() File "D:\Anaconda3\envs\testenv\lib\multiprocessing\connection.py", line 321, in _recv_bytes raise EOFError EOFError BrokenPipeError: [WinError 109] I run the code on win10 but this error occurred.
- Sync Vec porting
- After running the train.py, error "Unable to solve the normal equations in `LinearFeatureBaseline`. The matrix X^T*X (with X the design matrix) is not full-rank, regardless of the regularization (maximum regularization: 1.0)." occurs.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-maml-rl.