jumphone/beer

BEER: Batch EffEct Remover for single-cell data

BEER: Batch EffEct Remover for single-cell data

Environment: R

BEER's latest version: https://github.com/jumphone/BEER/releases

News:

• Mar. 2021 ( V0.1.9 ): First version for Seurat 4.0.0

• Feb. 2021 ( V0.1.8 ): Last version for Seurat 3.0.0

• Nov. 2019 ( v0.1.7 ): In ".simple_combine(D1, D2, FILL=TRUE)", "FILL" can help users to keep genes that are expressed in only one condition (fill the matrix with “0”). Default "FILL" is FALSE

• July 2019 ( v0.1.6 ): BEER can automatically adjust "GNUM" when cell number is small in some batch

• July 2019 ( v0.1.5 ): "ComBat" is used to replace "regression" of "ScaleData" (ComBat is much faster)

• July 2019 ( v0.1.4 ): Users can provide genes which need to be removed.

• July 2019 ( v0.1.3 ): Users can use VISA to extract peaks of scATAC-seq.

• ...

Content:

• Workflow
• Requirement (Installation)
• Vignettes (Usage)
• Reference
• License

---

Workflow:

Latest version

Please see V. Batch-effect Removal Enhancement for details of "Enhancement".

Requirement:

#R >=3.5
install.packages('Seurat') # ==4.0.0 

# Install ComBat:
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install("sva")
BiocManager::install("limma")

# Users can use "BEER" by directly importing "BEER.R" on the github webpage:

source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')

# Or, download and import it:

source('BEER.R')

For batch-effect removal enhancement, please install BBKNN: https://github.com/Teichlab/bbknn

Vignettes:

• I. Combine Two Batches
• II. Combine Multiple Batches
• III. UMAP-based Clustering
• IV. Combine scATAC-seq & scRNA-seq
• V. Batch-effect Removal Enhancement
• VI. Transfer Labels
• VII. Biological Interpretation
• VIII. QC before using BEER

Set Python

library(reticulate)
use_python("/home/toolkit/local/bin/python3",required=T)
py_config()

I. Combine Two Batches

Download demo data: https://github.com/jumphone/BEER/raw/master/DATA/demodata.zip

Please do basic quality control before using BEER (e.g. remove low-quality cells & genes).

For QC, please see: https://satijalab.org/seurat/v3.0/pbmc3k_tutorial.html

Step1. Load Data

library(Seurat)

source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')
#source('BEER.R')

#Read 10X data: pbmc.data <- Read10X(data.dir = "../data/pbmc3k/filtered_gene_bc_matrices/hg19/")

#Load Demo Data (subset of GSE70630: MGH53 & MGH54)
#Download: https://github.com/jumphone/BEER/raw/master/DATA/demodata.zip

D1 <- read.table(unz("demodata.zip","DATA1_MAT.txt"), sep='\t', row.names=1, header=T)
D2 <- read.table(unz("demodata.zip","DATA2_MAT.txt"), sep='\t', row.names=1, header=T)

# "D1" & "D2" are UMI matrix (or FPKM, RPKM, TPM, PKM ...; Should not be gene-centric scaled data)
# Rownames of "D1" & "D2" are gene names
# Colnames of "D1" & "D2" are cell names 

# There shouldn't be duplicated colnames in "D1" & "D2":
colnames(D1)=paste0('D1_', colnames(D1))
colnames(D2)=paste0('D2_', colnames(D2))

DATA=.simple_combine(D1,D2)$combine

# Users can use "DATA=.simple_combine(D1,D2, FILL=TRUE)$combine" to keep genes that are expressed in only one condition.

BATCH=rep('D2',ncol(DATA))
BATCH[c(1:ncol(D1))]='D1'

# Simple Quality Control (QC): check the number of sequenced genes
# PosN=apply(DATA,2,.getPos)
# USED=which(PosN>500 & PosN<4000) 
# DATA=DATA[,USED]; BATCH=BATCH[USED] 

Step2. Detect Batch Effect

mybeer=BEER(DATA, BATCH, GNUM=30, PCNUM=50, ROUND=1, GN=2000, SEED=1, COMBAT=TRUE, RMG=NULL)   

# DATA: Expression matrix. Rownames are genes. Colnames are cell names.
# BATCH: A character vector. Length is equal to the "ncol(DATA)".
# GNUM: the number of groups in each batch (default: 30)
# PCNUM: the number of computated PCA subspaces (default: 50)
# ROUND: batch-effect removal strength, positive integer (default: 1)
# GN: the number of variable genes in each batch (default: 2000)
# RMG: genes need to be removed (default: NULL)
# COMBAT: use ComBat to adjust expression value(default: TRUE)    

# Users can use "ReBEER" to adjust GNUM, PCNUM, ROUND, and RMG (it's faster than directly using BEER).
# mybeer <- ReBEER(mybeer, GNUM=30, PCNUM=50, ROUND=1, SEED=1, RMG=NULL) 

# Check selected PCs
PCUSE=mybeer$select
COL=rep('black',length(mybeer$cor))
COL[PCUSE]='red'
plot(mybeer$cor,mybeer$lcor,pch=16,col=COL,
    xlab='Rank Correlation',ylab='Linear Correlation',xlim=c(0,1),ylim=c(0,1))

Users can select PCA subspaces based on the distribution of "Rank Correlation" and "Linear Correlation".

# PCUSE=.selectUSE(mybeer, CUTR=0.7, CUTL=0.7, RR=0.5, RL=0.5)

Step3. Visualization

Keep batch effect:

pbmc_batch=CreateSeuratObject(counts = DATA, min.cells = 0, min.features = 0, project = "ALL") 
[email protected]$batch=BATCH
pbmc_batch=FindVariableFeatures(object = pbmc_batch, selection.method = "vst", nfeatures = 2000)   
VariableFeatures(object = pbmc_batch)
pbmc_batch <- NormalizeData(object = pbmc_batch, normalization.method = "LogNormalize", scale.factor = 10000)
pbmc_batch <- ScaleData(object = pbmc_batch, features = VariableFeatures(object = pbmc_batch))
pbmc_batch <- RunPCA(object = pbmc_batch, seed.use=123, npcs=50, features = VariableFeatures(object = pbmc_batch), ndims.print=1,nfeatures.print=1)
pbmc_batch <- RunUMAP(pbmc_batch, dims = 1:50, seed.use = 123,n.components=2)
DimPlot(pbmc_batch, reduction='umap', group.by='batch', pt.size=0.1) 

Remove batch effect:

pbmc <- mybeer$seurat
PCUSE <- mybeer$select
pbmc <- RunUMAP(object = pbmc, reduction='pca',dims = PCUSE, check_duplicates=FALSE)

DimPlot(pbmc, reduction='umap', group.by='batch', pt.size=0.1) 

II. Combine Multiple Batches

Download demo data: https://sourceforge.net/projects/beergithub/files/

Step1. Load Data

source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')

#Load Demo Data (Oligodendroglioma, GSE70630)
#Download: https://sourceforge.net/projects/beergithub/files/

D1=readRDS('MGH36.RDS')
D2=readRDS('MGH53.RDS')
D3=readRDS('MGH54.RDS')
D4=readRDS('MGH60.RDS')
D5=readRDS('MGH93.RDS')
D6=readRDS('MGH97.RDS')

BATCH=c(rep('D1',ncol(D1)),
        rep('D2',ncol(D2)),
        rep('D3',ncol(D3)),
        rep('D4',ncol(D4)),
        rep('D5',ncol(D5)),
        rep('D6',ncol(D6)) )
        
D12=.simple_combine(D1,D2)$combine
D34=.simple_combine(D3,D4)$combine
D56=.simple_combine(D5,D6)$combine
D1234=.simple_combine(D12,D34)$combine
D123456=.simple_combine(D1234,D56)$combine

DATA=D123456   

rm(D1);rm(D2);rm(D3);rm(D4);rm(D5);rm(D6)
rm(D12);rm(D34);rm(D56);rm(D1234);rm(D123456)

# Simple Quality Control (QC): check the number of sequenced genes
# PosN=apply(DATA,2,.getPos)
# USED=which(PosN>500 & PosN<4000) 
# DATA=DATA[,USED]; BATCH=BATCH[USED] 

Step2. Use BEER to Detect Batch Effect

mybeer=BEER(DATA, BATCH, GNUM=30, PCNUM=50, ROUND=1, GN=2000, SEED=1, COMBAT=TRUE )

# Check selected PCs
PCUSE=mybeer$select
COL=rep('black',length(mybeer$cor))
COL[PCUSE]='red'
plot(mybeer$cor,mybeer$lcor,pch=16,col=COL,
    xlab='Rank Correlation',ylab='Linear Correlation',xlim=c(0,1),ylim=c(0,1))

Step3. Visualization

Keep batch effect:

pbmc_batch=CreateSeuratObject(counts = DATA, min.cells = 0, min.features = 0, project = "ALL") 
[email protected]$batch=BATCH
pbmc_batch=FindVariableFeatures(object = pbmc_batch, selection.method = "vst", nfeatures = 2000)   
VariableFeatures(object = pbmc_batch)
pbmc_batch <- NormalizeData(object = pbmc_batch, normalization.method = "LogNormalize", scale.factor = 10000)
pbmc_batch <- ScaleData(object = pbmc_batch, features = VariableFeatures(object = pbmc_batch))
pbmc_batch <- RunPCA(object = pbmc_batch, seed.use=123, npcs=50, features = VariableFeatures(object = pbmc_batch), ndims.print=1,nfeatures.print=1)
pbmc_batch <- RunUMAP(pbmc_batch, dims = 1:50, seed.use = 123,n.components=2)
DimPlot(pbmc_batch, reduction='umap', group.by='batch', pt.size=0.1) 

Remove batch effect:

pbmc <- mybeer$seurat
PCUSE <- mybeer$select
pbmc <- RunUMAP(object = pbmc, reduction='pca',dims = PCUSE, check_duplicates=FALSE)

DimPlot(pbmc, reduction='umap', group.by='batch', pt.size=0.1)   

III. UMAP-based Clustering

VEC=pbmc@[email protected]

# Here, we use K-means to do the clustering
N=20
set.seed(123)
K=kmeans(VEC,centers=N)

CLUST=K$cluster
[email protected]$clust=as.character(CLUST)
DimPlot(pbmc, reduction='umap', group.by='clust', pt.size=0.5,label=TRUE)

# Or, manually select some cells

ppp=DimPlot(pbmc, reduction='umap', pt.size=0.5)
used.cells <- CellSelector(plot = ppp)

# Press "ESC"

markers <- FindMarkers(pbmc, ident.1=used.cells,only.pos=T)    
head(markers, n=20)

IV. Combine scATAC-seq & scRNA-seq

Please install "Signac": https://satijalab.org/signac/

Download DEMO data: https://sourceforge.net/projects/beer-file/files/ATAC/ & https://satijalab.org/signac/articles/pbmc_vignette.html

Step1. Load Data

source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')
#source('BEER.R')

library(Seurat)
library(Signac)
library(EnsDb.Hsapiens.v75)

counts <- Read10X_h5(filename = "./data/atac_v1_pbmc_10k_filtered_peak_bc_matrix.h5")

metadata <- read.csv(
  file = "./data/atac_v1_pbmc_10k_singlecell.csv",
  header = TRUE,
  row.names = 1
    )

chrom_assay <- CreateChromatinAssay(
    counts = counts,
    sep = c(":", "-"),
    genome = 'hg19',
    fragments = './data/atac_v1_pbmc_10k_fragments.tsv.gz',
    min.cells = 10,
    min.features = 200
   )

pbmc.atac <- CreateSeuratObject(
    counts = chrom_assay,
    assay = "peaks",
    meta.data = metadata
    )

annotations <- GetGRangesFromEnsDb(ensdb = EnsDb.Hsapiens.v75)
seqlevelsStyle(annotations) <- "UCSC"
genome(annotations) <- "hg19"
Annotation(pbmc.atac) <- annotations


gene.activities <- GeneActivity(pbmc.atac)
     
pbmc.rna <- readRDS("./data/pbmc_10k_v3.rds")

D1=as.matrix(gene.activities)
D2=as.matrix(pbmc.rna@assays$RNA@counts)
colnames(D1)=paste0('ATAC_', colnames(D1))
colnames(D2)=paste0('RNA_', colnames(D2))

D1=.check_rep(D1)
D2=.check_rep(D2)

DATA=.simple_combine(D1,D2)$combine
BATCH=rep('RNA',ncol(DATA))
BATCH[c(1:ncol(D1))]='ATAC'

Step2. Use BEER to Detect Batch Effect

mybeer <- BEER(DATA, BATCH, GNUM=30, PCNUM=50, ROUND=1, GN=5000, SEED=1, COMBAT=TRUE)
saveRDS(mybeer, file='mybeer')

# Users can use "ReBEER" to adjust parameters
mybeer <- ReBEER(mybeer, GNUM=100, PCNUM=100, ROUND=3, SEED=1)

PCUSE=mybeer$select
#PCUSE=.selectUSE(mybeer, CUTR=0.8, CUTL=0.8, RR=0.5, RL=0.5)

COL=rep('black',length(mybeer$cor))
COL[PCUSE]='red'
plot(mybeer$cor,mybeer$lcor,pch=16,col=COL,
    xlab='Rank Correlation',ylab='Linear Correlation',xlim=c(0,1),ylim=c(0,1))

Step3. Visualization

Keep batch effect:

pbmc_batch=CreateSeuratObject(counts = DATA, min.cells = 0, min.features = 0, project = "ALL") 
[email protected]$batch=BATCH
pbmc_batch=FindVariableFeatures(object = pbmc_batch, selection.method = "vst", nfeatures = 2000)   
VariableFeatures(object = pbmc_batch)
pbmc_batch <- NormalizeData(object = pbmc_batch, normalization.method = "LogNormalize", scale.factor = 10000)
pbmc_batch <- ScaleData(object = pbmc_batch, features = VariableFeatures(object = pbmc_batch))
pbmc_batch <- RunPCA(object = pbmc_batch, seed.use=123, npcs=50, features = VariableFeatures(object = pbmc_batch), ndims.print=1,nfeatures.print=1)
pbmc_batch <- RunUMAP(pbmc_batch, dims = 1:50, seed.use = 123,n.components=2)
DimPlot(pbmc_batch, reduction='umap', group.by='batch', pt.size=0.1)   

Remove batch effect:

pbmc <- mybeer$seurat  
PCUSE=mybeer$select
pbmc <- RunUMAP(object = pbmc, reduction='pca',dims = PCUSE, check_duplicates=FALSE)

DimPlot(pbmc, reduction='umap', group.by='batch', pt.size=0.1)    

[email protected]$celltype=rep(NA,length([email protected]$batch))
[email protected]$celltype[which([email protected]$batch=='RNA')][email protected]$celltype

DimPlot(pbmc, reduction='umap', group.by='celltype', pt.size=0.1,label=T)

saveRDS(mybeer, file='mybeer.final.RDS')

It's not good enough !

For further enhancement, please see V. Batch-effect Removal Enhancement.

V. Batch-effect Removal Enhancement

Please install BBKNN: https://github.com/Teichlab/bbknn

This DEMO follows IV. Combine scATAC-seq & scRNA-seq

source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')
#source('BEER.R')
mybeer=readRDS('mybeer.final.RDS')
pbmc.rna <- readRDS("./data/pbmc_10k_v3.rds")

Use ComBat & BBKNN without BEER:

pbmc <- mybeer$seurat
PCUSE=c(1:ncol(pbmc@[email protected]))
pbmc=BEER.combat(pbmc) #Adjust PCs using ComBat
umap=BEER.bbknn(pbmc, PCUSE, NB=3, NT=10)
pbmc@[email protected]=umap
DimPlot(pbmc, reduction='umap', group.by='batch', pt.size=0.1,label=F)

Use ComBat & BBKNN with BEER:

pbmc <- mybeer$seurat
PCUSE=mybeer$select   
pbmc=BEER.combat(pbmc) #Adjust PCs using ComBat
umap=BEER.bbknn(pbmc, PCUSE, NB=3, NT=10)
pbmc@[email protected]=umap
DimPlot(pbmc, reduction='umap', group.by='batch', pt.size=0.1,label=F)
 
saveRDS(pbmc, file='seurat.enh.RDS')

[email protected]$celltype=rep(NA,length([email protected]$batch))
[email protected]$celltype[which([email protected]$batch=='RNA')][email protected]$celltype
DimPlot(pbmc, reduction='umap', group.by='celltype', pt.size=0.1,label=T)

Use BBKNN in Python:

Please download beer_bbknn.py.

source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')
#source('BEER.R')
pbmc <- mybeer$seurat
pbmc=BEER.combat(pbmc) #Adjust PCs using ComBat
PCUSE = mybeer$select
used.pca = pbmc@[email protected][,PCUSE]
.writeTable(DATA=used.pca, PATH='used.pca.txt',SEP=',')
.writeTable([email protected]$batch, PATH='batch.txt',SEP=',')

Then, use "beer_bbknn.py" in your command line (please modify parameters in beer_bbknn.py):

python beer_bbknn.py

Finally, load the output of beer_bbknn.py and draw UMAP:

umap=read.table('bbknn_umap.txt',sep='\t',header=FALSE)
umap=as.matrix(umap)
rownames(umap)=rownames(pbmc@[email protected])
colnames(umap)=colnames(pbmc@[email protected])
pbmc@[email protected]=umap
DimPlot(pbmc, reduction='umap', group.by='batch', pt.size=0.1,label=F)

VI. Transfer labels

This DEMO follows V. Batch-effect Removal Enhancement

[email protected]$celltype=rep(NA,length([email protected]$batch))
[email protected]$celltype[which([email protected]$batch=='RNA')][email protected]$celltype
#DimPlot(pbmc, reduction='umap', group.by='celltype', pt.size=0.1,label=T)

#######
VEC=pbmc@[email protected]
set.seed(123)
N=150
K=kmeans(VEC,centers=N)
[email protected]$kclust=K$cluster   
#DimPlot(pbmc, reduction='umap', group.by='kclust', pt.size=0.1,label=T)

[email protected]$transfer=rep(NA, length([email protected]$celltype))
TMP=cbind([email protected]$celltype, [email protected]$kclust)

KC=unique([email protected]$kclust)
i=1
while(i<=length(KC)){
    this_kc=KC[i]
    this_index=which([email protected]$kclust==this_kc)
    this_tb=table([email protected]$celltype[this_index])
    if(length(this_tb)!=0){
        this_ct=names(this_tb)[which(this_tb==max(this_tb))[1]]
        [email protected]$transfer[this_index]=this_ct}
    i=i+1}
    
[email protected][email protected]$celltype
NA.index=which(is.na([email protected]$celltype))
[email protected]$tf.ct[NA.index][email protected]$transfer[NA.index]

######
RNA.cells=colnames(pbmc)[which([email protected]$batch=='RNA')]
ATAC.cells=colnames(pbmc)[which([email protected]$batch=='ATAC')]

library(ggplot2)

plot.all <- DimPlot(pbmc, reduction='umap', group.by='batch', 
    pt.size=0.1,label=F) + labs(title = "Batches")

plot.ct <- DimPlot(pbmc,reduction='umap', group.by='tf.ct', 
    pt.size=0.1,label=T) + labs(title = "CellType")

plot.rna <- DimPlot(pbmc, cells=RNA.cells,reduction='umap', 
    group.by='tf.ct', pt.size=0.1,label=T,plot.title='RNA.transfer') + labs(title = "RNA")

plot.atac <- DimPlot(pbmc, cells=ATAC.cells,reduction='umap', 
    group.by='tf.ct', pt.size=0.1,label=T,plot.title='ATAC.transfer') + labs(title = "ATAC")

CombinePlots(list(all=plot.all, ct=plot.ct, rna=plot.rna, atac=plot.atac))

If you want to visualize peak signals of any given cluster, please go to https://github.com/jumphone/VISA.

VII. Biological Interpretation

Please install "RITANdata" and "RITAN".

RITAN: https://bioconductor.org/packages/devel/bioc/vignettes/RITAN/inst/doc/enrichment.html

This DEMO follows IV. Combine scATAC-seq & scRNA-seq

library(RITANdata)
library(RITAN)

PCUSE <- mybeer$select
PCALL <- c(1:length(mybeer$cor))
PCnotUSE <- PCALL[which(!PCALL %in% PCUSE)]

LD=mybeer$seurat@[email protected]
GNAME=rownames(LD)

N=100
getPosAndNegTop <- function(x){
    O=c(order(x)[1:N],order(x)[(length(x)-(N-1)):length(x)])
    G=GNAME[O]
    return(G)
    }

GMAT=apply(LD,2,getPosAndNegTop)
colnames(GMAT)=paste0(colnames(GMAT),'_R_',round(mybeer$cor,1),"_L_",round(mybeer$lcor,1))
GMAT=toupper(GMAT)

GMAT=GMAT[,PCnotUSE]
#GMAT=GMAT[,PCUSE]

study_set=list()
TAG=colnames(GMAT)
i=1
while(i<=ncol(GMAT)){
     study_set=c(study_set,list(GMAT[,i]))
     i=i+1
     }  
     
names(study_set)=TAG
#names(geneset_list)
resources=c('KEGG_filtered_canonical_pathways','MSigDB_Hallmarks')

e <- term_enrichment_by_subset( study_set, q_value_threshold = 1e-5, 
                            resources = resources,
                            all_symbols = cached_coding_genes )

plot( e, show_values = FALSE, label_size_y = 7, label_size_x = 7, cap=10 )

VIII. QC before using BEER

Download demo data: https://sourceforge.net/projects/beergithub/files/

Step1. Load Data

source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')

#Load Demo Data (Oligodendroglioma, GSE70630)
#Download: https://sourceforge.net/projects/beergithub/files/

D1=readRDS('MGH36.RDS')
D2=readRDS('MGH53.RDS')
D3=readRDS('MGH54.RDS')
D4=readRDS('MGH60.RDS')
D5=readRDS('MGH93.RDS')
D6=readRDS('MGH97.RDS')

BATCH=c(rep('D1',ncol(D1)),
        rep('D2',ncol(D2)),
        rep('D3',ncol(D3)),
        rep('D4',ncol(D4)),
        rep('D5',ncol(D5)),
        rep('D6',ncol(D6)) )
    
D12=.simple_combine(D1,D2)$combine
D34=.simple_combine(D3,D4)$combine
D56=.simple_combine(D5,D6)$combine
D1234=.simple_combine(D12,D34)$combine
D123456=.simple_combine(D1234,D56)$combine

DATA=D123456   

rm(D1);rm(D2);rm(D3);rm(D4);rm(D5);rm(D6)
rm(D12);rm(D34);rm(D56);rm(D1234);rm(D123456)

Step2. QC

pbmc <- CreateSeuratObject(counts = DATA, project = "pbmc3k", min.cells = 0, min.features = 0)
Idents(pbmc)=BATCH
[email protected]$batch=BATCH

pbmc <- subset(pbmc, subset = nFeature_RNA > 200 & nFeature_RNA < 2500 & percent.mt < 5)

Please fllow https://satijalab.org/seurat/v3.1/pbmc3k_tutorial.html to do Quality Control.

[email protected]$batch

DATA=as.matrix(pbmc@assays$RNA@counts[,which(colnames(pbmc@assays$RNA@counts) %in% colnames(pbmc@assays$RNA@data))])

Step3. BEER

Refer to II. Combine Multiple Batches

Reference:

Feng Zhang, Yu Wu, Weidong Tian*; A novel approach to remove the batch effect of single-cell data, Cell Discovery, 2019, https://doi.org/10.1038/s41421-019-0114-x

Differences between the latest version and the manuscript version

Latest version: https://github.com/jumphone/BEER/releases

Manuscript version: https://github.com/jumphone/BEER/archive/0.0.2.zip

---

More tools & studies: https://fzhang.bioinfo-lab.com/

License

MIT License

Copyright (c) 2019 Zhang, Feng

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

beer's Issues

Error in BEER(DATA, BATCH, GNUM = 30, PCNUM = 50, ROUND = 1, GN = 2000, : no slot of name "data" for this object of class "Assay5"

Thany you for the very nice package, but I've met some trouble, could you please offer some help?
Here I downloaded the demodata, and run the code as the README file says,

> mybeer=BEER(DATA, BATCH, GNUM=30, PCNUM=50, ROUND=1, GN=2000, SEED=1, COMBAT=TRUE, RMG=NULL)

And I receive the error as

[1] "BEER start!"
[1] "2024-01-23 22:34:55 CST"
[1] "Group number (GNUM) is:"
[1] 30
[1] "Varible gene number (GN) of each batch is:"
[1] 2000
[1] "ROUND is:"
[1] 1
[1] 1
[1] "D1"
Warning: Data is of class data.frame. Coercing to dgCMatrix.
Normalizing layer: counts
Performing log-normalization
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Finding variable features for layer counts
Calculating gene variances
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
[1] 2
[1] "D2"
Warning: Data is of class data.frame. Coercing to dgCMatrix.
Normalizing layer: counts
Performing log-normalization
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Finding variable features for layer counts
Calculating gene variances
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
[1] "Total varible gene number (GN) is:"
[1] 2551
Warning: Data is of class data.frame. Coercing to dgCMatrix.
Normalizing layer: counts
Performing log-normalization
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Error in BEER(DATA, BATCH, GNUM = 30, PCNUM = 50, ROUND = 1, GN = 2000, :
no slot of name "data" for this object of class "Assay5"

I'm sure I followed the pipeline as the README file says. I may make a stupid mistake, but I finally failed to correct it. Here is the session info:
> sessionInfo()
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=Chinese (Simplified)_China.utf8 LC_CTYPE=Chinese (Simplified)_China.utf8
[3] LC_MONETARY=Chinese (Simplified)_China.utf8 LC_NUMERIC=C
[5] LC_TIME=Chinese (Simplified)_China.utf8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] stringi_1.8.3 reticulate_1.34.0 Seurat_5.0.1 SeuratObject_5.0.1
[5] sp_2.1-2 limma_3.54.2 sva_3.46.0 BiocParallel_1.32.6
[9] genefilter_1.80.3 mgcv_1.9-1 nlme_3.1-164 dbplyr_2.4.0.9000

loaded via a namespace (and not attached):
[1] spam_2.10-0 plyr_1.8.9 igraph_1.6.0 lazyeval_0.2.2
[5] splines_4.2.2 RcppHNSW_0.5.0 listenv_0.9.0 scattermore_1.2
[9] GenomeInfoDb_1.34.9 ggplot2_3.4.4 digest_0.6.34 htmltools_0.5.7
[13] fansi_1.0.5 magrittr_2.0.3 memoise_2.0.1 tensor_1.5
[17] cluster_2.1.6 ROCR_1.0-11 globals_0.16.2 Biostrings_2.66.0
[21] annotate_1.76.0 matrixStats_1.2.0 spatstat.sparse_3.0-3 colorspace_2.1-0
[25] blob_1.2.4 ggrepel_0.9.5 dplyr_1.1.3 crayon_1.5.2
[29] RCurl_1.98-1.14 jsonlite_1.8.8 progressr_0.14.0 spatstat.data_3.0-4
[33] survival_3.5-7 zoo_1.8-12 glue_1.6.2 polyclip_1.10-6
[37] gtable_0.3.4 zlibbioc_1.44.0 XVector_0.38.0 leiden_0.4.3.1
[41] future.apply_1.11.1 BiocGenerics_0.44.0 abind_1.4-5 scales_1.3.0
[45] DBI_1.2.1 edgeR_3.40.2 spatstat.random_3.2-2 miniUI_0.1.1.1
[49] Rcpp_1.0.11 viridisLite_0.4.2 xtable_1.8-4 bit_4.0.5
[53] dotCall64_1.1-1 stats4_4.2.2 htmlwidgets_1.6.4 httr_1.4.7
[57] RColorBrewer_1.1-3 ellipsis_0.3.2 ica_1.0-3 pkgconfig_2.0.3
[61] XML_3.99-0.16 uwot_0.1.16 deldir_2.0-2 locfit_1.5-9.8
[65] utf8_1.2.4 tidyselect_1.2.0 rlang_1.1.2 reshape2_1.4.4
[69] later_1.3.2 AnnotationDbi_1.60.2 munsell_0.5.0 tools_4.2.2
[73] cachem_1.0.8 cli_3.6.1 generics_0.1.3 RSQLite_2.3.4
[77] ggridges_0.5.5 stringr_1.5.1 fastmap_1.1.1 goftest_1.2-3
[81] bit64_4.0.5 fitdistrplus_1.1-11 purrr_1.0.2 RANN_2.6.1
[85] KEGGREST_1.38.0 pbapply_1.7-2 future_1.33.1 mime_0.12
[89] compiler_4.2.2 rstudioapi_0.15.0 plotly_4.10.4 png_0.1-8
[93] spatstat.utils_3.0-4 tibble_3.2.1 RSpectra_0.16-1 lattice_0.21-9
[97] Matrix_1.6-3 vctrs_0.6.4 pillar_1.9.0 lifecycle_1.0.4
[101] BiocManager_1.30.22 spatstat.geom_3.2-7 lmtest_0.9-40 RcppAnnoy_0.0.21
[105] data.table_1.14.10 cowplot_1.1.2 bitops_1.0-7 irlba_2.3.5.1
[109] httpuv_1.6.13 patchwork_1.2.0 R6_2.5.1 promises_1.2.1
[113] KernSmooth_2.23-22 gridExtra_2.3 IRanges_2.32.0 parallelly_1.36.0
[117] codetools_0.2-19 fastDummies_1.7.3 MASS_7.3-60.0.1 sctransform_0.4.1
[121] S4Vectors_0.36.2 GenomeInfoDbData_1.2.9 parallel_4.2.2 grid_4.2.2
[125] tidyr_1.3.0 Rtsne_0.17 spatstat.explore_3.2-5 Biobase_2.58.0
[129] shiny_1.8.0

Thank you very much for your help.

Merge multiple scATAC Seq objects

Hello,

Thanks for the great package! I would like to merge multiple scATAC Seq objects with BEER and BBKNN that I am currently analyzing with Signac, so they are Seurat objects. How would one do this?

Input data

It is unclear in the vignette which data is used as input.

From the demo data, I can see that the input is a data frame ( https://github.com/jumphone/BEER/raw/master/DATA/demodata.zip )

Using a Seurat object, I can use the .simple_combine function to merge assays using the [["RNA"]] slot:

DATA=.simple_combine( Seurat_1 [["RNA"]], Seurat_2 [["RNA"]] )

The result is a sparse Matrix of class "dgCMatrix".

But impossible to use the FILL=T, I have the following error:

Error in rbind2(argl[[i]], r) : 
  no method for coercing this S4 class to a vector
Called from: rbind2(argl[[i]], r)

1. Do I use the .simple_combine function properly?
2. What can I do concerning the FILL=T option (I actually should have complete gene record)

Error in asMethod(object) : Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 105

Hello,

Thank you so much for developing so great software!

If the size of the file is not too big, BEER works well.

If too big, BEER does not work.

Thank you in advance for your great help!

Best,

Yue

> library(Seurat)
Collecting package metadata (current_repodata.json): ...working... failed

UnavailableInvalidChannel: The channel is not accessible or is invalid.
  channel name: anaconda/cloud/pytorch
  channel url: https://mirrors.cloud.tencent.com/anaconda/cloud/pytorch
  error code: 404

You will need to adjust your conda configuration to proceed.
Use `conda config --show channels` to view your configuration's current state,
and use `conda config --show-sources` to view config file locations.


> library(limma)
> library(sva)
Loading required package: mgcv
Loading required package: nlme
This is mgcv 1.8-31. For overview type 'help("mgcv-package")'.
Loading required package: genefilter
Loading required package: BiocParallel
> source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')
[1] "Welcome to BEER (v0.1.7)!"
> library(data.table)
data.table 1.12.8 using 6 threads (see ?getDTthreads).  Latest news: r-datatable.com
> getwd()
[1] "/home/li"
> setwd("/home/li/Transcriptomic_patients/donor")
> a<-"donor_82_01.csv"
> b<-"donor_85_02.csv"
> c<-"donor_87_03.csv"
> d<-"donor_89_04.csv"
> e<-"donor_91_05.csv"
> f<-"donor_93_06.csv"
> g<-"donor_95_07.csv"
> h<-"donor_97_08.csv"
> a1<-data.frame(fread(a),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> b1<-data.frame(fread(b),check.names=FALSE, row.names=1,header=TRUE)
> c1<-data.frame(fread(c),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> d1<-data.frame(fread(d),check.names=FALSE, row.names=1,header=TRUE)
> e1<-data.frame(fread(e),check.names=FALSE, row.names=1,header=TRUE)
> f1<-data.frame(fread(f),check.names=FALSE, row.names=1,header=TRUE)
> g1<-data.frame(fread(g),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> h1<-data.frame(fread(h),check.names=FALSE, row.names=1,header=TRUE)
> setwd("/home/li/Transcriptomic_patients/patients")
> i<-"mild_69_141.csv"
> j<-"mild_70_142.csv"
> k<-"severe_71_143.csv"
> l<-"mild_72_144.csv"
> m<-"severe_73_145.csv"
> n<-"severe_74_146.csv"
> o<-"severe_51_148.csv"
> p<-"severe_52_149.csv"
> q<-"severe_53_152.csv"
> i1<-data.frame(fread(i),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> j1<-data.frame(fread(j),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> k1<-data.frame(fread(k),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> l1<-data.frame(fread(l),check.names=FALSE, row.names=1,header=TRUE)
> m1<-data.frame(fread(m),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> n1<-data.frame(fread(n),check.names=FALSE, row.names=1,header=TRUE)
> o1<-data.frame(fread(o),check.names=FALSE, row.names=1,header=TRUE)
> p1<-data.frame(fread(p),check.names=FALSE, row.names=1,header=TRUE)
> q1<-data.frame(fread(q),check.names=FALSE, row.names=1,header=TRUE)
|--------------------------------------------------|
|==================================================|
> D1<-cbind(a1,b1)
> D2<-cbind(i1,j1)
> colnames(D1)=paste0('D1_', colnames(D1))
> colnames(D2)=paste0('D2_', colnames(D2))
> DATA=.simple_combine(D1,D2)$combine
> BATCH=rep('D2',ncol(DATA))
> BATCH[c(1:ncol(D1))]='D1'
> PosN=apply(DATA,2,.getPos)
> USED=which(PosN>500 & PosN<4000) 
> DATA=DATA[,USED]; BATCH=BATCH[USED] 
> mybeer=BEER(DATA, BATCH, GNUM=30, PCNUM=50, ROUND=1, GN=2000, SEED=1, COMBAT=TRUE, RMG=NULL)   
[1] "BEER start!"
[1] "2020-05-27 20:15:01 PDT"
[1] "Group number (GNUM) is:"
[1] 30
[1] "Varible gene number (GN) of each batch is:"
[1] 2000
[1] "ROUND is:"
[1] 1
[1] 1
[1] "D1"
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variances
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
[1] 2
[1] "D2"
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variances
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
[1] "Total varible gene number (GN) is:"
[1] 3101
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Found2batches
Adjusting for0covariate(s) or covariate level(s)
Standardizing Data across genes
Fitting L/S model and finding priors
Finding parametric adjustments
Adjusting the Data

Centering and scaling data matrix
  |======================================================================| 100%
[1] "Calculating PCs ..."
PC_ 1 
Positive:  ENSG00000011600 
Negative:  ENSG00000196260 
Warning: The default method for RunUMAP has changed from calling Python UMAP via reticulate to the R-native UWOT using the cosine metric
To use Python UMAP via reticulate, set umap.method to 'umap-learn' and metric to 'correlation'
This message will be shown once per session
20:16:09 UMAP embedding parameters a = 0.9922 b = 1.112
20:16:09 Read 13570 rows and found 50 numeric columns
20:16:09 Using Annoy for neighbor search, n_neighbors = 30
20:16:09 Building Annoy index with metric = cosine, n_trees = 50
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
20:16:12 Writing NN index file to temp file /tmp/RtmpDd8OkR/fileabb1332a96ba
20:16:12 Searching Annoy index using 1 thread, search_k = 3000
20:16:17 Annoy recall = 100%
20:16:18 Commencing smooth kNN distance calibration using 1 thread
20:16:19 Initializing from normalized Laplacian + noise
20:16:21 Commencing optimization for 200 epochs, with 596834 positive edges
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
20:16:29 Optimization finished
[1] "Get group for:"
[1] "D1"
[1] "Group Number:"
[1] 30
[1] "Get group for:"
[1] "D2"
[1] "Group Number:"
[1] 30
[1] "Finding MN pairs..."
[1] "1 / 60"
[1] "ROUND:"
[1] 1
[1] "Number of MN pairs:"
[1] 4
[1] "Evaluating PCs ..."
[1] "Start"
[1] 1
[1] 2
[1] 3
[1] 4
[1] 5
[1] 6
[1] 7
[1] 8
[1] 9
[1] 10
[1] 11
[1] 12
[1] 13
[1] 14
[1] 15
[1] 16
[1] 17
[1] 18
[1] 19
[1] 20
[1] 21
[1] 22
[1] 23
[1] 24
[1] 25
[1] 26
[1] 27
[1] 28
[1] 29
[1] 30
[1] 31
[1] 32
[1] 33
[1] 34
[1] 35
[1] 36
[1] 37
[1] 38
[1] 39
[1] 40
[1] 41
[1] 42
[1] 43
[1] 44
[1] 45
[1] 46
[1] 47
[1] 48
[1] 49
[1] 50
[1] "Finished!!!"
[1] "############################################################################"
[1] "BEER cheers !!! All main steps finished."
[1] "############################################################################"
[1] "2020-05-27 20:16:52 PDT"
> D1<-cbind(a1,b1,c1,d1,e1,f1,g1,h1)
> D2<-cbind(i1,j1,k1,l1,m1,n1,o1,p1,q1)
> colnames(D1)=paste0('D1_', colnames(D1))
> colnames(D2)=paste0('D2_', colnames(D2))
> DATA=.simple_combine(D1,D2)$combine
> BATCH=rep('D2',ncol(DATA))
> BATCH[c(1:ncol(D1))]='D1'
> PosN=apply(DATA,2,.getPos)
> USED=which(PosN>500 & PosN<4000) 
> DATA=DATA[,USED]; BATCH=BATCH[USED] 
> mybeer=BEER(DATA, BATCH, GNUM=30, PCNUM=50, ROUND=1, GN=2000, SEED=1, COMBAT=TRUE, RMG=NULL)   
[1] "BEER start!"
[1] "2020-05-27 20:34:03 PDT"
[1] "Group number (GNUM) is:"
[1] 30
[1] "Varible gene number (GN) of each batch is:"
[1] 2000
[1] "ROUND is:"
[1] 1
[1] 1
[1] "D1"
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variances
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
[1] 2
[1] "D2"
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variances
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
[1] "Total varible gene number (GN) is:"
[1] 3060
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Error in asMethod(object) : 
  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 105

Error in names(x) <- value : 'names' attribute [1] must be the same length as the vector [0]

Hello,

Thank you for developing so nice software!

I can go through the demodata of DATA2_MAT.txt.

I have problem when upload my data: donor_82_01.csv .

Thank you in advance for great help!

Best,

Yue

> D1<-read.csv(file='donor_82_01.csv', sep='\t',row.names=1, header=T)
> D2<-read.csv(file='donor_85_02.csv', sep='\t',row.names=1, header=T)
> colnames(D1)=paste0('D1_', colnames(D1))
Error in names(x) <- value : 
  'names' attribute [1] must be the same length as the vector [0]

donor_82_01.zip

unused argument (reduction.use = "umap")

Hello,

Thank you for developing so nice software "BEER".

I followed the: https://github.com/jumphone/BEER

Thanks in advance for any great help!

Best,

Yue

> DimPlot(pbmc_batch, reduction.use='umap', group.by='batch', pt.size=0.1) Error in DimPlot(pbmc_batch, reduction.use = "umap", group.by = "batch", : unused argument (reduction.use = "umap")

Error in `contrasts<-`(`*tmp*`, value = contr.funs[1 + isOF[nn]]) : contrasts can be applied only to factors with 2 or more levels

Hello,
Thank you for developing so nice software!

Thank you in advance for your great help!

Best,

Yue

> D1 <- read.table(file='donor.csv', sep=',', row.names=1, header=TRUE,as.is=T)
> D2 <- read.table(file='patients.csv', sep=',', row.names=1, header=TRUE,as.is=T)
> colnames(D1)=paste0('D1_', colnames(D1))
> colnames(D2)=paste0('D2_', colnames(D2))
> source('https://raw.githubusercontent.com/jumphone/BEER/master/BEER.R')
> DATA=.simple_combine(D1,D2)$combine
> BATCH=rep('D2',ncol(DATA))
> BATCH[c(1:ncol(D1))]='D1'
> PosN=apply(DATA,2,.getPos)
> USED=which(PosN>500 & PosN<4000)
>  DATA=DATA[,USED]; BATCH=BATCH[USED] 
> mybeer=BEER(DATA, BATCH, GNUM=30, PCNUM=50, ROUND=1, GN=2000, SEED=1, COMBAT=TRUE, RMG=NULL) 
[1] "BEER start!"
[1] "2020-05-26 10:31:19 PDT"
Loading required package: stringi
[1] "Group number (GNUM) is:"
[1] 30
[1] "Varible gene number (GN) of each batch is:"
[1] 2000
[1] "ROUND is:"
[1] 1
[1] 1
[1] "D2"
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating gene variances
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
[1] "Total varible gene number (GN) is:"
[1] 2000
Performing log-normalization
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Error in `contrasts<-`(`*tmp*`, value = contr.funs[1 + isOF[nn]]) : 
  contrasts can be applied only to factors with 2 or more levels
In addition: Warning messages:
1: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  at  -0.78561
2: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  radius  5.5604e-05
3: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  all data on boundary of neighborhood. make span bigger
4: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  pseudoinverse used at -0.78561
5: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  neighborhood radius 0.0074568
6: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  reciprocal condition number  1
7: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  zero-width neighborhood. make span bigger
8: In simpleLoess(y, x, w, span, degree = degree, parametric = parametric,  :
  There are other near singularities as well. 0.090619