First of all, thanks for making the framework of twoWayOne2One work. It accelerates scaled-up simulations significantly compared to twoWayMPI.
I was testing the twoWayOne2One scheme (in the 21.11 version) and found that the chemical source and heat transfer terms pushed from LIGGGHTS were accumulating. In theory, they should be reset to zero when entering LIGGGHTS loops. The root cause is that twoWayOne2One does not synchronize the latestpush time as twoWayMPI does. I will explain the finding below:
The following code snippet from fix_cfd_coupling_chemistry.cpp shows the field resetting during the initial_integrate.
void FixCfdCouplingChemistry::initial_integrate(int)
{
bigint prev_time = update->ntimestep - 1;
int *mask = atom->mask;
int nlocal = atom->nlocal;
for (int k = 0; k < num_species; ++k)
{
if (prev_time == fix_coupling_->latestpush(mod_spec_names_[k]))
{
for (int i = 0; i < nlocal; ++i)
{
if (mask[i] & groupbit)
fix_masschange_[k]->vector_atom[i] = 0.;
}
}
}
However, when the twoWayOne2One is triggered. The if statement if (prev_time == fix_coupling_->latestpush(mod_spec_names_[k])) is never true because latestpush time is not updated. It is always -1 as the value it is initialized. Digging deeper it can be found that the latestpush time can be updated in cfd_datacoupling.cpp
void CfdDatacoupling::push(const char *name, const char *type, void *&, const char *, int iworld)
{
// for MPI this is called by the library interface
// check if the requested property was registered by a LIGGGHTS model
// ie this checks if the model settings of OF and LIGGGHTS fit together
int found = 0;
for(int i = 0; i < npush_; i++)
{
if(strcmp(name,pushnames_[i]) == 0 && strcmp(type,pushtypes_[i]) == 0)
{
found = 1;
pushinvoked_[i] = 1;
// TL:
latestpush_[i] = update->ntimestep;
}
...
So using twoWayOne2One this push function is never called. To help myself understand why I first look into how it is done in twoWayMPI. This push subroutine is called in the following order
- subModels/dataExchangeModel/twoWayMPI/twoWayMPI.C:
in getData calls data_liggghts_to_of
- LIGGGHTS/src/library_cfd_coupling.cpp:
in data_liggghts_to_of calls data_liggghts_to_of_universe
- LIGGGHTS/src/library_cfd_coupling.cpp:
in data_liggghts_to_of_universe calls fcfd->get_dc()->push
- LIGGGHTS/src/cfd_datacoupling_mpi.cpp:
in push calls CfdDatacoupling::push
such that the latestpush time is updated.
My Solution
My solution is basically following the same logic as twoWayMPI. When every time a set of data is pushed to OpenFOAM, trigger this CfdDatacoupling::push so that the latestpush time is updated.
- subModels/dataExchangeModel/twoWayOne2One/twoWayOne2One.C:
in getData calls a new function register_push_liggghts_to_of
void twoWayOne2One::getData
(
const word& name,
const word& type,
double ** const& field,
label /*step*/
) const
{
if (name == "x") // the location is transferred by couple()
{
return;
}
if (type == "vector-atom")
{
double **tmp_= static_cast<double **>(lammps_extract_atom(lmp,name.c_str()));
...
...
lig2foam_->exchange<double>
(
tmp_,
lig2foam_vec_tmp_,
3
);
extractCollected<double>
(
lig2foam_vec_tmp_,
const_cast<double**&>(field),
3
);
// let liggghts know field is pushed such that latestPush() is updated // C.C. Modified
register_push_liggghts_to_of
(
name.c_str(),
type.c_str(),
lmp,
(void*&) field,
"double"
);
...
- LIGGGHTS/src/library_cfd_coupling.cpp
in register_push_liggghts_to_of calls function dc->push
void register_push_liggghts_to_of
(
const char *name,
const char *type,
void *ptr,
void *&data,
const char* datatype
)
{
FixCfdCoupling* fcfd = (FixCfdCoupling*)locate_coupling_fix(ptr);
CfdDatacouplingOne2One* dc = static_cast<CfdDatacouplingOne2One*>(fcfd->get_dc());
dc->push(name, type, data, datatype);
}
- LIGGGHTS/src/cfd_datacoupling_one2one.cpp:
remove the error message
void CfdDatacouplingOne2One::push(const char *name,const char *type,void *&to,const char *datatype)
{
CfdDatacoupling::push(name,type,to,datatype);
//error->one(FLERR,"Illegal call to CfdDatacouplingOne2One::pull, use with twoWayOne2One");
}
I am not sure about why the error message is original here and what is the consequence to MPI after calling this push.
With this new implementation, the chemical source and heat transfer terms are no longer accumulating over time. They are reset to zero at the first loops of initial_integrate. The latestpull time can also be updated in the similar manner.
I test it with my case and surprisingly the results are exactly the same as that of using twoWayMPI.
Hopefully my report can help solve the trivial issue. I am pretty sure there is another way to update the latestpush time. My solution is just how I tackled this issue. Looking forward to the new release.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent