I'm a web development engineer
I like Java
modouxiansheng / doraemon Goto Github PK
View Code? Open in Web Editor NEW对于一些后端常用框架的深入理解 An in-depth understanding of some common back-end frameworks
对于一些后端常用框架的深入理解 An in-depth understanding of some common back-end frameworks
I'm a web development engineer
I like Java
项目pull 下来 启动 rpc这个项目启动不起来 找不到 :common 这个模块
打不可链接
你好,我现在有一个问题请教一下,我使用的是lettuce,把您这个项目改造之后无法使用,请问有这方面的建议么
您好,我想咨询您关于Redis 热点key的问题;
对于热点key的处理,应该什么时候去进行处理呢。是当某个redis的节点的那些性能指标出现下降的时候,在去做热点key的统计,然后在做处理吗?
1.热点key的处理时机?
2.判断处理热点key,应该考虑什么性能指标?
废话就不多说了,直接上代码了。
`package com.rayn.commons.io;
import org.apache.commons.lang3.time.StopWatch;
import java.io.*;
import java.nio.ByteBuffer;
import java.nio.MappedByteBuffer;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.channels.Pipe;
import java.nio.channels.WritableByteChannel;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
/**
@author Ryan
@Date 2019/11/16 11:51
*/
public class IOStreamTurningExample {
public IOStreamTurningExample() {
}
/**
Native IO Stream
@param srcFile
@param destFile
*/
public static void nativeStreamWriterFile(File srcFile, File destFile) {
final StopWatch stopWatch = new StopWatch();
stopWatch.start();
try (ZipOutputStream zipOutputStream = new ZipOutputStream(new FileOutputStream(destFile))) {
try (final FileInputStream fileInputStream = new FileInputStream(srcFile)) {
zipOutputStream.putNextEntry(new ZipEntry(destFile.getName()));
byte[] b = new byte[1024 * 1024 * 5];
int readSize = 0;
while ((readSize = fileInputStream.read(b)) > 0) {
zipOutputStream.write(b, 0, readSize);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
stopWatch.stop();
printInfo("nativeStreamWriterFile", stopWatch);
}
/**
Use Buffer IO Stream
@param srcFile
@param destFile
*/
public static void bufferStreamWriterFile(File srcFile, File destFile) {
final StopWatch stopWatch = new StopWatch();
stopWatch.start();
try (ZipOutputStream zipOutputStream = new ZipOutputStream(new FileOutputStream(destFile));
final BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(zipOutputStream)) {
try (final BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(srcFile))) {
zipOutputStream.putNextEntry(new ZipEntry(destFile.getName()));
byte[] b = new byte[1024 * 1024 * 5];
int readSize = 0;
while ((readSize = bufferedInputStream.read(b)) != -1) {
bufferedOutputStream.write(b, 0, readSize);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
stopWatch.stop();
printInfo("bufferStreamWriterFile", stopWatch);
}
/**
Use Channel IO Stream
@param srcFile
@param destFile
*/
public static void channelStreamWriterFile(File srcFile, File destFile) {
final long fileSize = srcFile.length();
final StopWatch stopWatch = new StopWatch();
stopWatch.start();
try (ZipOutputStream zipOutputStream = new ZipOutputStream(new FileOutputStream(destFile));
final WritableByteChannel writableByteChannel = Channels.newChannel(zipOutputStream)) {
try (final FileChannel fileChannel = new FileInputStream(srcFile).getChannel()) {
zipOutputStream.putNextEntry(new ZipEntry(destFile.getName()));
fileChannel.transferTo(0, fileSize, writableByteChannel);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
stopWatch.stop();
printInfo("channelStreamWriterFile", stopWatch);
}
/**
@param srcFile
@param destFile
*/
public static void mappedStreamWriterFile(File srcFile, File destFile) {
final long fileSize = srcFile.length();
final StopWatch stopWatch = new StopWatch();
stopWatch.start();
try (ZipOutputStream zipOutputStream = new ZipOutputStream(new FileOutputStream(destFile));
final WritableByteChannel writableByteChannel = Channels.newChannel(zipOutputStream)) {
zipOutputStream.putNextEntry(new ZipEntry(destFile.getName()));
try (final FileChannel fileChannel = new RandomAccessFile(srcFile, "r").getChannel()) {
final MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileSize);
writableByteChannel.write(mappedByteBuffer);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
stopWatch.stop();
printInfo("mappedStreamWriterFile", stopWatch);
}
/**
*
*/
public static void pipeStreamWriterFile(File srcFile, File destFile) {
final long fileSize = srcFile.length();
final StopWatch stopWatch = new StopWatch();
stopWatch.start();
try (ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(destFile));
final WritableByteChannel writableByteChannel = Channels.newChannel(zos)) {
zos.putNextEntry(new ZipEntry(destFile.getName()));
final Pipe pipe = Pipe.open();
CompletableFuture.runAsync(() -> {
try (ZipOutputStream zipOutputStream1 = new ZipOutputStream(Channels.newOutputStream(pipe.sink()));
final WritableByteChannel writableByteChannel1 = Channels.newChannel(zipOutputStream1)) {
zipOutputStream1.putNextEntry(new ZipEntry(destFile.getName()));
final FileChannel destFileChannel = new FileInputStream(srcFile).getChannel();
destFileChannel.transferTo(0, fileSize, writableByteChannel1);
destFileChannel.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
});
final Pipe.SourceChannel sourceChannel = pipe.source();
final ByteBuffer buffer = ByteBuffer.allocate(1024 * 1024 * 10);
while (sourceChannel.read(buffer) >= 0) {
buffer.flip();
writableByteChannel.write(buffer);
buffer.clear();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
stopWatch.stop();
printInfo("pipeStreamWriterFile", stopWatch);
}
private static void printInfo(String method, StopWatch stopWatch) {
System.out.println("\n\n" + method + ":");
System.out.println("Start Time : " + stopWatch.getStartTime());
System.out.println("Duration Time : " + stopWatch.toString());
}
public static void main(String[] args) {
File src_file = new File("D:/test/bigfile.mp4");
File dest_file = new File("D:/test/bigfile.zip");
nativeStreamWriterFile(src_file, dest_file);
bufferStreamWriterFile(src_file, dest_file);
channelStreamWriterFile(src_file, dest_file);
mappedStreamWriterFile(src_file, dest_file);
pipeStreamWriterFile(src_file, dest_file);
try {
TimeUnit.SECONDS.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
`
唯一与作者不同之处在于 fileInputStream.read(new byte[1024 * 1024 * 5) 。
测试 srcFile 大小为:1.13 G 的电影。
测试结果
nativeStreamWriterFile:
Start Time : 1573884268436
Duration Time : 0:00:52.334
bufferStreamWriterFile:
Start Time : 1573884320778
Duration Time : 0:00:51.496
channelStreamWriterFile:
Start Time : 1573884372275
Duration Time : 0:00:52.157
mappedStreamWriterFile:
Start Time : 1573884424432
Duration Time : 0:00:51.618
pipeStreamWriterFile:
Start Time : 1573884476051
Duration Time : 0:01:19.865
请问,是我哪里的姿势不对吗?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.