SR好像有一些bug

[Fu-Dayuan]

SR好像有一些bug

[chang-github-00]

What is the index of this sample in Jericho?
Also, wonder if it is possible to provide us with a full trajectory (all 17 turns of action) so that we could reimplement this error.

[Fu-Dayuan]

mistral-7b-v0.1
{
"id": 12,
"task_name": "zork2",
"goal": "You are an explorer in the underground world. Your quest is to find a unicorn in the underground maze. You will be needing light to avoid being eaten by Grue in the darkness.",
"difficulty": "easy",
"is_done": true,
"progress_rate": 0.25,
"grounding_acc": 1.0,
"score_change_record": [
[
3,
0.25
]
],
"trajectory": {
"Interaction Turn 0": {
"Goal": "You are an explorer in the underground world. Your quest is to find a unicorn in the underground maze. You will be needing light to avoid being eaten by Grue in the darkness.",
"Observation": "Taken. ",
"Action": "Take brass lantern",
"Progress Rate": 0
},
"Interaction Turn 1": {
"Action": "Examine sword of Elvish workmanship",
"Observation": "I don't know the word "workmanship". ",
"Progress Rate": 0
},
"Interaction Turn 2": {
"Action": "Check valid actions",
"Observation": "You can take the following actions: wait, look around sword, turn on lamp, take sword, put down lamp, look at sword with lamp, put lamp on ground, take sword from ground, throw lamp at sword, turn on lamp with all, south, check valid actions",
"Progress Rate": 0
},
"Interaction Turn 3": {
"Action": "Turn on lamp",
"Observation": "The lamp is now on. ",
"Progress Rate": 0.25
},
"Interaction Turn 4": {
"Action": "Take sword of Elendir",
"Observation": "I don't know the word "elendir". ",
"Progress Rate": 0.25
},
"Interaction Turn 5": {
"Action": "You can take the following actions: wait",
"Observation": "I don't know the word "you". ",
"Progress Rate": 0.25
},
"Interaction Turn 6": {
"Action": "Check valid actions",
"Observation": "You can take the following actions: wait, look around sword, put out lamp, take sword, put down lamp, look at sword with lamp, put lamp on ground, take sword from ground, throw lamp at sword, south, check valid actions",
"Progress Rate": 0.25
},
"Interaction Turn 7": {
"Action": "Put out lamp",
"Observation": "The lamp is now off. ",
"Progress Rate": 0.25
},
"Interaction Turn 8": {
"Action": "Take sword of Elendir",
"Observation": "I don't know the word "elendir". ",
"Progress Rate": 0.25
},
"Interaction Turn 9": {
"Action": "Look around sword",
"Observation": "Inside the Barrow You are inside an ancient barrow hidden deep within a dark forest. The barrow opens into a narrow tunnel at its southern end. You can see a faint glow at the far end. A sword of Elvish workmanship is on the ground. A strange little man in a long cloak appears suddenly in the room. He is wearing a high pointed hat embroidered with astrological signs. He has a long, stringy, and unkempt beard. The Wizard draws forth his wand and waves it in your direction. It begins to glow with a faint blue glow. The Wizard, in a deep and resonant voice, speaks the word "Fear!" He cackles gleefully. You look at the Wizard in terror. You scramble away, trying to get as far as possible from him. Narrow Tunnel You are standing at the southern end of a narrow tunnel where it opens into a wide cavern. The cavern is dimly illuminated by phosphorescent mosses clinging to its high ceiling. A deep ravine winds through the cavern, with a small stream at the bottom. The walls of the ravine are steep and crumbly. A foot bridge crosses the ravine to the south. ",
"Progress Rate": 0.25
},
"Interaction Turn 10": {
"Action": "Look around sword",
"Observation": "You can't see any sword here! ",
"Progress Rate": 0.25
},
"Interaction Turn 11": {
"Action": "Look around",
"Observation": "Narrow Tunnel You are standing at the southern end of a narrow tunnel where it opens into a wide cavern. The cavern is dimly illuminated by phosphorescent mosses clinging to its high ceiling. A deep ravine winds through the cavern, with a small stream at the bottom. The walls of the ravine are steep and crumbly. A foot bridge crosses the ravine to the south. ",
"Progress Rate": 0.25
},
"Interaction Turn 12": {
"Action": "Check valid actions",
"Observation": "You can take the following actions: jump, look around bridge, turn on lamp, cross bridge, jump across bridge, put down lamp, south, look at bridge with ravine, throw lamp at bridge, turn on lamp with all, pour lamp from bridge, north, check valid actions",
"Progress Rate": 0.25
},
"Interaction Turn 13": {
"Action": "Look around bridge",
"Observation": "Narrow Tunnel You are standing at the southern end of a narrow tunnel where it opens into a wide cavern. The cavern is dimly illuminated by phosphorescent mosses clinging to its high ceiling. A deep ravine winds through the cavern, with a small stream at the bottom. The walls of the ravine are steep and crumbly. A foot bridge crosses the ravine to the south. ",
"Progress Rate": 0.25
},
"Interaction Turn 14": {
"Action": "Check valid actions",
"Observation": "You can take the following actions: jump, look around bridge, turn on lamp, cross bridge, jump across bridge, put down lamp, south, look at bridge with ravine, throw lamp at bridge, turn on lamp with all, pour lamp from bridge, north, check valid actions",
"Progress Rate": 0.25
},
"Interaction Turn 15": {
"Action": "Look around bridge",
"Observation": "Narrow Tunnel You are standing at the southern end of a narrow tunnel where it opens into a wide cavern. The cavern is dimly illuminated by phosphorescent mosses clinging to its high ceiling. A deep ravine winds through the cavern, with a small stream at the bottom. The walls of the ravine are steep and crumbly. A foot bridge crosses the ravine to the south. ",
"Progress Rate": 0.25
},
"Interaction Turn 16": {
"Action": "Check valid actions",
"Observation": "You can take the following actions: jump, look around bridge, turn on lamp, cross bridge, jump across bridge, put down lamp, south, look at bridge with ravine, throw lamp at bridge, turn on lamp with all, pour lamp from bridge, north, check valid actions",
"Progress Rate": 0.25
},
"Interaction Turn 17": {
"Action": "Look around bridge",
"Observation": "Narrow Tunnel You are standing at the southern end of a narrow tunnel where it opens into a wide cavern. The cavern is dimly illuminated by phosphorescent mosses clinging to its high ceiling. A deep ravine winds through the cavern, with a small stream at the bottom. The walls of the ravine are steep and crumbly. A foot bridge crosses the ravine to the south. ",
"Progress Rate": 0.25
},
"Interaction Turn 18": {
"Action": "Check valid actions",
"Observation": "You can take the following actions: jump, look around bridge, turn on lamp, cross bridge, jump across bridge, put down lamp, south, look at bridge with ravine, throw lamp at bridge, turn on lamp with all, pour lamp from bridge, north, check valid actions",
"Progress Rate": 0.25
},
"Interaction Turn 19": {
"Action": "Look around bridge",
"Observation": "Narrow Tunnel You are standing at the southern end of a narrow tunnel where it opens into a wide cavern. The cavern is dimly illuminated by phosphorescent mosses clinging to its high ceiling. A deep ravine winds through the cavern, with a small stream at the bottom. The walls of the ravine are steep and crumbly. A foot bridge crosses the ravine to the south. ",
"Progress Rate": 0.25
},
"Interaction Turn 20": {
"Action": "Check valid actions",
"Observation": "You can take the following actions: jump, look around bridge, turn on lamp, cross bridge, jump across bridge, put down lamp, south, look at bridge with ravine, throw lamp at bridge, turn on lamp with all, pour lamp from bridge, north, check valid actions",
"Progress Rate": 0.25
},
"Interaction Turn 21": {
"Action": "Look around bridge",
"Observation": "Narrow Tunnel You are standing at the southern end of a narrow tunnel where it opens into a wide cavern. The cavern is dimly illuminated by phosphorescent mosses clinging to its high ceiling. A deep ravine winds through the cavern, with a small stream at the bottom. The walls of the ravine are steep and crumbly. A foot bridge crosses the ravine to the south. ",
"Progress Rate": 0.25
},
"Interaction Turn 22": {
"Action": "You can take the following actions: jump",
"Observation": "I don't know the word "you". ",
"Progress Rate": 0.25
},
"Interaction Turn 23": {
"Action": "You can take the following actions: jump",
"Observation": "I don't know the word "you". ",
"Progress Rate": 0.25
},
"Interaction Turn 24": {
"Action": "jump",
"Observation": "This was not a very safe place to try jumping. In the movies, your life would be passing before your eyes. **** You have died **** Now, let's take a look here... Well, you probably deserve another chance. I can't quite fix you up completely, but you can't have everything. Room of Red Mist You are inside a huge crystalline sphere filled with thin red mist. The mist becomes blue to the west. You strain to look out through the mist... You see a small room with a sign on the wall, but it is too blurry to read. ",
"Progress Rate": 0.25
}
}
}
{
"id": 13,
"task_name": "zork3",
"goal": "You need to find your way into a secret passage. Your best chance is to bribe the old gatekeeper at Engravings room with some bread. You need to search for bread somewhere.",
"difficulty": "easy",
"is_done": true,
"progress_rate": 0.0,
"grounding_acc": 1.0,
"score_change_record": [],
"trajectory": {
"Interaction Turn 0": {
"Goal": "You need to find your way into a secret passage. Your best chance is to bribe the old gatekeeper at Engravings room with some bread. You need to search for bread somewhere.",
"Observation": "The lamp is turned off. ",
"Action": "examine brass lantern",
"Progress Rate": 0
},
"Interaction Turn 1": {
"Action": "turn on the lantern",
"Observation": "(Taken) The lamp is now on. ",
"Progress Rate": 0
},
"Interaction Turn 2": {
"Action": "south",
"Observation": "Junction You are at the junction of a north-south passage and an east-west passage. To the north, you can make out the bottom of a stairway. The ways to the east and south are relatively cramped, but a wider trail leads to the west. Standing before you is a great rock. Imbedded within it is an Elvish sword. ",
"Progress Rate": 0
},
"Interaction Turn 3": {
"Action": "take the sword",
"Observation": "The sword is deeply imbedded within the rock. You can't even begin to budge it. ",
"Progress Rate": 0
},
"Interaction Turn 4": {
"Action": "examine the rock",
"Observation": "The great rock contains: A sword ",
"Progress Rate": 0
},
"Interaction Turn 5": {
"Action": "examine the sword",
"Observation": "I see nothing special about the sword. ",
"Progress Rate": 0
},
"Interaction Turn 6": {
"Action": "take the sword",
"Observation": "The sword is deeply imbedded within the rock. You can't even begin to budge it. ",
"Progress Rate": 0
},
"Interaction Turn 7": {
"Action": "check valid actions",
"Observation": "You can take the following actions: turn off lamp, put down lamp, west, throw west at lamp, north, south, east, check valid actions",
"Progress Rate": 0
},
"Interaction Turn 8": {
"Action": "turn off the lamp",
"Observation": "The lamp is now off. It is now pitch black. ",
"Progress Rate": 0
},
"Interaction Turn 9": {
"Action": "put down the sword",
"Observation": "It's too dark to see. ",
"Progress Rate": 0
},
"Interaction Turn 10": {
"Action": "west",
"Observation": "Oh, no! A lurking grue slithered into the room and devoured you! **** You have died **** You find yourself deep within the earth in a barren prison cell. Outside the iron-barred window, you can see a great, fiery pit. Flames leap up and very nearly sear your flesh. After a while, footfalls can be heard in the distance, then closer and closer.... The door swings open, and in walks an old man. He is dressed simply in a hood and cloak, wearing a few simple jewels, carrying something under one arm, and leaning on a wooden staff. A single key, as if to a massive prison cell, hangs from his belt. He raises the staff toward you and you hear him speak, as if in a dream: "I await you, though your journey be long and full of peril. Go then, and let me not wait long!" You feel some great power well up inside you and you fall to the floor. The next moment, you are awakening, as if from a deep slumber. Endless Stair There is a lamp here. ",
"Progress Rate": 0
}
}
}
{
"id": 14,
"task_name": "detective",
"goal": "Search for the culprit around town. He is rumored to be staying at holiday inn. Beware of suspicious people and get a weapon and use it.",
"difficulty": "hard",
"is_done": true,
"progress_rate": 0.0,
"grounding_acc": 1.0,
"score_change_record": [],
"trajectory": {
"Interaction Turn 0": {
"Goal": "Search for the culprit around town. He is rumored to be staying at holiday inn. Beware of suspicious people and get a weapon and use it.",
"Observation": " It is a piece of white paper. ",
"Action": "examine paper",
"Progress Rate": 0
},
"Interaction Turn 1": {
"Action": "read paper",
"Observation": " CONFIDENTIAL: Detective was created by Matt Barringer. He has worked hard on this so you better enjoy it. I did have fun making it though. But I'd REALLY appreciate it if you were kind enough to send a postcard or... dare I even say it?... money... to: Matt Barringer 325 Olive Ave Piedmont CA 94611 Just tell me if you like it or not. If you want to talk to me over a BBS call the Ghostbuster Central BBS at [phone]. There is an Exile Games file area. Have fun. I WILL give hints out over the BBS to any of my games. ",
"Progress Rate": 0
},
"Interaction Turn 2": {
"Action": "north",
"Observation": " << Outside >> You are outside in the cold. To the east is a dead end. To the west is the rest of the street. Papers are blowing around. It's amazingly cold for this time of year. ",
"Progress Rate": 0
},
"Interaction Turn 3": {
"Action": "check valid actions",
"Observation": "You can take the following actions: north, west, check valid actions",
"Progress Rate": 0
},
"Interaction Turn 4": {
"Action": "west",
"Observation": " << Outside >> You are still on the streets. To the north is a restraunt where the mayor ate often. To the east is the Mayor's home. ",
"Progress Rate": 0
},
"Interaction Turn 5": {
"Action": "north",
"Observation": " << Restraunt >> You are about to enter the restraunt when two guys jump you. They take your wallet and beat you a bit. Then you flash your badge and that riles them. Your body was discovered in a river 10 miles away. *** You have died *** In that game you scored 20 out of a possible 360, in 5 turns. Would you like to RESTART, RESTORE a saved game, give the FULL score for that game or QUIT?",
"Progress Rate": 0
}
}
}