xzymustbexzy/chain-of-experts

Official implementation of the paper "Chain-of-Experts: When LLMs Meet Complex Operation Research Problems"

chain-of-experts's Issues

Inquiry About the Release Date of the Formal 37 ComplexOR Datasets

Hello,

In the README, it's noted that "the ComplexOR dataset is still in the review stage, and a raw version has been uploaded, with the formal 37 datasets mentioned in the paper to be released soon."

Could you please provide any updates on when the formal 37 datasets might be available for public access? Understanding the timeline would be greatly beneficial for planning my research activities around this data.

Thank you for your groundbreaking work and looking forward to your response.

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

配置了DeepSeek API （LangChain 支持 OpenAI API 接口，而 DeepSeek API 接口与 OpenAI 兼容。）测试接口已能正确调用
运行python run_exp.py --dataset LPWP --problem "prob_.*" --algorithm coe命令
出现以下提示：

/opt/conda/envs/CoE/bin/python run_exp.py --dataset LPWP --problem prob_.* --algorithm coe
/opt/conda/envs/CoE/lib/python3.10/site-packages/langchain/callbacks/init.py:37: LangChainDeprecationWarning: Importing this callback from langchain is deprecated. Importing it from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.callbacks import get_openai_callback.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
/opt/conda/envs/CoE/lib/python3.10/site-packages/langchain/init.py:29: UserWarning: Importing PromptTemplate from langchain root module is no longer supported. Please use langchain.prompts.PromptTemplate instead.
warnings.warn(
/opt/conda/envs/CoE/lib/python3.10/site-packages/langchain/init.py:29: UserWarning: Importing OpenAI from langchain root module is no longer supported. Please use langchain_community.llms.OpenAI instead.
warnings.warn(
/opt/conda/envs/CoE/lib/python3.10/site-packages/langchain/init.py:29: UserWarning: Importing LLMChain from langchain root module is no longer supported. Please use langchain.chains.LLMChain instead.
warnings.warn(
/opt/conda/envs/CoE/lib/python3.10/site-packages/langchain/chat_models/init.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.chat_models import ChatOpenAI.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
0%| | 0/288 [00:00<?, ?it/s]Save log to log/run_coe_LPWP_1717666151
/opt/conda/envs/CoE/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The class ChatOpenAI was deprecated in LangChain 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run pip install -U langchain-openai and import as from langchain_openai import ChatOpenAI.
warn_deprecated(
Choose next expert: Modeling Expert
Input
Now the origin problem is as follow:
A toy store decides to deliver gifts using two shipping companies, a new one and an old one. The new company can deliver 50 gifts per trip while the old company can deliver 70 gifts per trip. The new company uses 30 liters of diesel per trip while the old company uses 40 liters of diesel per trip. The toy store needs to deliver at least 1000 gifts. There can be at most 15 trips made by the new company. In order to make sure that the old company does not go out of business, at least 40% of all trips must be made by the old company. How many trips should each company make to minimize the total amount of diesel used?
And the comments from other experts are as follow:
There is no comment available, please ignore this section.

Give your MIP model of this problem. Additionally, please note that your model needs to be a solvable linear programming model or a mixed-integer programming model. This also means that the expressions of the constraint conditions can only be equal to, greater than or equal to, or less than or equal to (> or < are not allowed to appear and should be replaced to be \geq or \leq).

Your output format should be a JSON like this:
{
"VARIABLES": "A mathematical description about variables",
"CONSTRAINS": "A mathematical description about constrains",
"OBJECTIVE": "A mathematical description about objective"
}

Given comment:

{
    "VARIABLES": "Let x be the number of trips made by the new company, and y be the number of trips made by the old company.",
    "CONSTRAINS": "1. x * 50 + y * 70 >= 1000 (Total gifts delivered must be at least 1000) \n2. x <= 15 (At most 15 trips can be made by the new company) \n3. y >= 0.4 * (x + y) (At least 40% of all trips must be made by the old company) \n4. x >= 0, y >= 0 (Non-negativity constraints)",
    "OBJECTIVE": "Minimize D = 30x + 40y (Total diesel used, where 30 and 40 are the diesel consumption rates per trip for the new and old companies respectively)"
}

Choose next expert: Parameter Extractor
Input
As a parameter extraction expert, your role is to identify and extract the relevant variables, constrans, objective from the problem statement.
Your expertise in the problem domain will help in accurately identifying and describing these variables.
Please review the problem description and provide the extracted variables along with their definitions:
A toy store decides to deliver gifts using two shipping companies, a new one and an old one. The new company can deliver 50 gifts per trip while the old company can deliver 70 gifts per trip. The new company uses 30 liters of diesel per trip while the old company uses 40 liters of diesel per trip. The toy store needs to deliver at least 1000 gifts. There can be at most 15 trips made by the new company. In order to make sure that the old company does not go out of business, at least 40% of all trips must be made by the old company. How many trips should each company make to minimize the total amount of diesel used?

And the comments from other experts are as follow:
Modeling Expert: ``````json
{
"VARIABLES": "Let x be the number of trips made by the new company, and y be the number of trips made by the old company.",
"CONSTRAINS": "1. x * 50 + y * 70 >= 1000 (Total gifts delivered must be at least 1000) \n2. x <= 15 (At most 15 trips can be made by the new company) \n3. y >= 0.4 * (x + y) (At least 40% of all trips must be made by the old company) \n4. x >= 0, y >= 0 (Non-negativity constraints)",
"OBJECTIVE": "Minimize D = 30x + 40y (Total diesel used, where 30 and 40 are the diesel consumption rates per trip for the new and old companies respectively)"
}

Please note that the information you extract is for the purpose of modeling, which means your variables, constraints, and objectives need to meet the requirements of a solvable LP or MIP model. Within the constraints, the comparison operators must be equal to, greater than or equal to, or less than or equal to (> or < are not allowed to appear and should be replaced to be \geq or \leq).


Given comment:
Based on the problem description and the comments from the Modeling Expert, here are the extracted variables, constraints, and objective function, formatted to meet the requirements of a solvable LP or MIP model:

**Variables:**
- `x`: Number of trips made by the new company.
- `y`: Number of trips made by the old company.

**Constraints:**
1. `x * 50 + y * 70 >= 1000` (Total gifts delivered must be at least 1000)
2. `x <= 15` (At most 15 trips can be made by the new company)
3. `y >= 0.4 * (x + y)` (At least 40% of all trips must be made by the old company)
4. `x >= 0` (Non-negativity constraint for the new company)
5. `y >= 0` (Non-negativity constraint for the old company)

**Objective:**
- `Minimize D = 30x + 40y` (Total diesel used, where 30 and 40 are the diesel consumption rates per trip for the new and old companies respectively)

These variables, constraints, and objective function are structured to be compatible with linear programming (LP) or mixed-integer programming (MIP) models, ensuring that all constraints use appropriate comparison operators (`>=`, `<=`, `=`) and that the objective function is to be minimized.
Choose next expert: Terminology Interpreter
Input
As a domain knowledge terminology interpreter, your role is to provide additional information and insights related to the problem domain. 
Here are some relevant background knowledge about this problem: None. 

You can contribute by sharing your expertise, explaining relevant concepts, and offering suggestions to improve the problem understanding and formulation. 
Please provide your input based on the given problem description: 
A toy store decides to deliver gifts using two shipping companies, a new one and an old one. The new company can deliver 50 gifts per trip while the old company can deliver 70 gifts per trip. The new company uses 30 liters of diesel per trip while the old company uses 40 liters of diesel per trip. The toy store needs to deliver at least 1000 gifts. There can be at most 15 trips made by the new company. In order to make sure that the old company does not go out of business, at least 40% of all trips must be made by the old company. How many trips should each company make to minimize the total amount of diesel used?

Your output format should be a JSON like this (choose at most 3 hardest terminology):
[
  {
    "terminology": "...",
    "interpretation": "..."
  }
]


Traceback (most recent call last):
  File "/opt/test/Chain-of-Experts/run_exp.py", line 113, in <module>
    main()
  File "/opt/test/Chain-of-Experts/run_exp.py", line 67, in main
    answer = chain_of_experts(
  File "/opt/test/Chain-of-Experts/main.py", line 75, in chain_of_experts
    comment_text = next_expert.forward(problem, comment_pool)#获取专家的评论
  File "/opt/test/Chain-of-Experts/experts/terminology_interpreter.py", line 121, in forward
    output = json.loads(output)
  File "/opt/conda/envs/CoE/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/opt/conda/envs/CoE/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/conda/envs/CoE/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
  0%|          | 0/288 [00:45<?, ?it/s]
  是什么问题导致：
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)  

Conductor选择出的专家总是错误

如图是conductor.py的调试结果，预测的专家总是存在***，导致每次只能随机选择。

LPWP and ComplexOR dataset

Could you please add the LPWP and ComplexOR dataset? It is a little bit hard to run the code without them.

Guidance on Integrating New Benchmarks with Chain-of-experts

Hello,

I am currently assessing the performance of Chain-of-experts using new benchmarks, specifically MAMO and IndustryOR. I observed that for existing benchmarks like NL4OPT and ComplexOR, you have implemented few-shot examples.

Is the creation of few-shot examples mandatory for integrating any new benchmarks? If so, could you provide guidance or suggest any modifications necessary to adapt your pipeline to accommodate these new benchmarks?

Thank you for your support!