Java GenericObjectPool 对象池化技术之SpringBoot sftp 连接池工具类详解

2023-05-14 20:05:17 对象 连接池 详解

Java BasePooledObjectFactory 对象池化技术

通常一个对象创建、销毁非常耗时的时候，我们不会频繁的创建和销毁它，而是考虑复用。复用对象的一种做法就是对象池，将创建好的对象放入池中维护起来，下次再用的时候直接拿池中已经创建好的对象继续用，这就是池化的思想。

Apache Commons Pool是一个对象池的框架，他提供了一整套用于实现对象池化的api。它提供了三种对象池：GenericKeyedObjectPool，SoftReferenceObjectPool和GenericObjectPool，其中GenericObjectPool是我们最常用的对象池，内部实现也最复杂。

GenericObjectPool

GenericObjectPool 是一个通用对象池框架，我们可以借助它实现一个健壮的对象池，UML图如下所示：

GenericObjectPool 实现了ObjectPool接口，而ObjectPool就是对象池的核心接口，它定义了一个对象池应该实现的行为。

public interface ObjectPool<T> extends Closeable {
    
    T borrowObject() throws Exception, NoSuchElementException, IllegalStateException;
    
    void returnObject(T var1) throws Exception;
    
    void invalidateObject(T var1) throws Exception;

    
    void addObject() throws Exception, IllegalStateException, UnsupportedOperationException;
    
    int getNumIdle();
    
    int getNuMactive();
    
    void clear() throws Exception, UnsupportedOperationException;
    
    void close();
}

BasePooledObjectFactory

Java BasePooledObjectFactory 对象池化技术

使用GenericObjectPool只需要创建一个对象工厂类，继承BasePooledObjectFactory并重写它的create()和destroyObject()。
如下文中的：SftpPool.java

public interface PooledObjectFactory<T> {
    
    PooledObject<T> makeObject() throws Exception;

    
    void destroyObject(PooledObject<T> var1) throws Exception;

    
    boolean validateObject(PooledObject<T> var1);

    
    void activateObject(PooledObject<T> var1) throws Exception;

    
    void passivateObject(PooledObject<T> var1) throws Exception;
}

配置类GenericObjectPoolConfig

GenericObjectPoolConfig是封装GenericObject池配置的简单“结构”，此类不是线程安全的；它仅用于提供创建池时使用的属性。大多数情况，可以使用GenericObjectPoolConfig提供的默认参数就可以满足日常的需求。

工作原理流程

  • 构造方法
    当我们执行构造方法时，主要工作就是创建了一个存储对象的LinkedList类型容器，也就是概念意义上的“池”
  • 从对象池中获取对象
    获取池中的对象是通过borrowObject()命令，源码比较复杂，简单而言就是去LinkedList中获取一个对象，如果不存在的话，要调用构造方法中第一个参数Factory工厂类的makeObject()方法去创建一个对象再获取，获取到对象后要调用validateObject方法判断该对象是否是可用的，如果是可用的才拿去使用。LinkedList容器减一
  • 归还对象到线程池
    简单而言就是先调用validateObject方法判断该对象是否是可用的，如果可用则归还到池中，LinkedList容器加一，如果是不可以的则调用destroyObject方法进行销毁

上面三步就是最简单的流程，由于取和还的流程步骤都在borrowObject和returnObject方法中固定的，所以我们只要重写Factory工厂类的makeObject()和validateObject以及destroyObject方法即可实现最简单的池的管理控制，通过构造方法传入该Factory工厂类对象则可以创建最简单的对象池管理类。这算是比较好的解耦设计模式，借和还的流程如下图所示：

使用Demo

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
    <version>2.7.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
<dependency>
    <groupId>com.jcraft</groupId>
    <artifactId>jsch</artifactId>
    <version>0.1.55</version>
</dependency>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="Http://Maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>vipsoft-parent</artifactId>
        <groupId>com.vipsoft.boot</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>vipsoft-sftp</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>

        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-pool2</artifactId>
            <version>2.7.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
        <dependency>
            <groupId>com.jcraft</groupId>
            <artifactId>jsch</artifactId>
            <version>0.1.55</version>
        </dependency>

        <dependency>
            <groupId>org.eclipse.paho</groupId>
            <artifactId>org.eclipse.paho.client.MQttv3</artifactId>
            <version>1.2.5</version>
        </dependency>


        <dependency>
            <groupId>cn.hutool</groupId>
            <artifactId>hutool-all</artifactId>
            <version>5.3.6</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-WEB</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

application.yaml

server:
  port: 8088
  application:
    name: sftp Demo


sftp:
  host: 172.16.3.88 # 服务器ip
  port: 22 # ssh端口
  username: root # 用户名
  passWord: root # 密码
  # 连接池参数
  pool:
    max-total: 10
    max-idle: 10
    min-idle: 5

SftpPoolException.java

package com.vipsoft.sftp.exception;



public class SftpPoolException extends RuntimeException {

    private static final long serialVersionUID = 1L;

    
    public SftpPoolException() {
    }

    
    public SftpPoolException(String message) {
        super(message);
    }

    
    public SftpPoolException(String message, Throwable cause) {
        super(message, cause);
    }

    
    public SftpPoolException(Throwable cause) {
        super(cause);
    }

    
    public SftpPoolException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
        super(message, cause, enableSuppression, writableStackTrace);
    }

}

config

SftpConfig.java

package com.vipsoft.sftp.config;

import com.vipsoft.sftp.pool.SftpFactory;
import com.vipsoft.sftp.pool.SftpPool;
import com.vipsoft.sftp.utils.SftpUtil;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableConfigurationProperties(SftpProperties.class)
public class SftpConfig {
    // 工厂
    @Bean
    public SftpFactory sftpFactory(SftpProperties properties) {
        return new SftpFactory(properties);
    }

    // 连接池
    @Bean
    public SftpPool sftpPool(SftpFactory sftpFactory) {
        return new SftpPool(sftpFactory);
    }

    // 辅助类
    @Bean
    public SftpUtil sftpUtil(SftpPool sftpPool) {
        return new SftpUtil(sftpPool);
    }
}

SftpProperties.java

package com.vipsoft.sftp.config;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "sftp")
public class SftpProperties {

    private String host;
    private int port = 22;
    private String username = "root";
    private String password = "root";
    private Pool pool = new Pool();

    public String getHost() {
        return host;
    }

    public void setHost(String host) {
        this.host = host;
    }

    public int getPort() {
        return port;
    }

    public void setPort(int port) {
        this.port = port;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    public Pool getPool() {
        return pool;
    }

    public void setPool(Pool pool) {
        this.pool = pool;
    }

    public static class Pool extends GenericObjectPoolConfig<ChannelSftp> {

        private int maxTotal = DEFAULT_MAX_TOTAL;
        private int maxIdle = DEFAULT_MAX_IDLE;
        private int minIdle = DEFAULT_MIN_IDLE;

        public Pool() {
            super();
        }
        @Override
        public int getMaxTotal() {
            return maxTotal;
        }
        @Override
        public void setMaxTotal(int maxTotal) {
            this.maxTotal = maxTotal;
        }
        @Override
        public int getMaxIdle() {
            return maxIdle;
        }
        @Override
        public void setMaxIdle(int maxIdle) {
            this.maxIdle = maxIdle;
        }
        @Override
        public int getMinIdle() {
            return minIdle;
        }
        @Override
        public void setMinIdle(int minIdle) {
            this.minIdle = minIdle;
        }
    }
}

Pool

SftpFactory.java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.vipsoft.sftp.config.SftpProperties;
import com.vipsoft.sftp.exception.SftpPoolException;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Properties;

public class SftpFactory extends BasePooledObjectFactory<ChannelSftp> {

    private  final Logger logger = LoggerFactory.getLogger(this.getClass());

    private SftpProperties properties;

    public SftpProperties getProperties() {
        return properties;
    }

    public void setProperties(SftpProperties properties) {
        this.properties = properties;
    }

    public SftpFactory(SftpProperties properties) {
        this.properties = properties;
    }

    @Override
    public ChannelSftp create() {
        try {
            JSch jsch = new JSch();
            Session sshSession = jsch.getSession(properties.getUsername(), properties.getHost(), properties.getPort());
            sshSession.setPassword(properties.getPassword());
            Properties sshConfig = new Properties();
            sshConfig.put("StrictHosTKEyChecking", "no");
            sshSession.setConfig(sshConfig);
            sshSession.connect();
            ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp");
            channel.connect();
            return channel;
        } catch (JSchException e) {
            throw new SftpPoolException("连接sfpt失败", e);
        }
    }

    @Override
    public PooledObject<ChannelSftp> wrap(ChannelSftp channelSftp) {
        return new DefaultPooledObject<>(channelSftp);
    }

    // 销毁对象
    @Override
    public void destroyObject(PooledObject<ChannelSftp> p) {
        ChannelSftp channelSftp = p.getObject();
        channelSftp.disconnect();
    }
}

SftpPool.java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPool;

public class SftpPool<T> extends GenericObjectPool<ChannelSftp> {

    public SftpPool(SftpFactory factory) {
        super(factory,factory.getProperties().getPool());
    }

    
    @Override
    public ChannelSftp borrowObject() throws Exception {
        return super.borrowObject();
    }

    
    @Override
    public void returnObject(ChannelSftp channelSftp) {
        if (channelSftp!=null) {
            super.returnObject(channelSftp);
        }
    }

}

Utils

ByteUtil.java

package com.vipsoft.sftp.utils;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.SftpException;
import com.vipsoft.sftp.exception.SftpPoolException;
import com.vipsoft.sftp.pool.SftpPool;

import java.io.InputStream;

public class SftpUtil {

    private SftpPool pool;

    public SftpUtil(SftpPool pool) {
        this.pool = pool;
    }

    
    public byte[] download(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            InputStream in = sftp.get(name);
            return ByteUtil.inputStreamToByteArray(in);
        } catch (Exception e) {
            throw new SftpPoolException("sftp下载文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    
    public void upload(String dir, String name, InputStream in) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            mkdirs(sftp, dir);
            sftp.cd(dir);
            sftp.put(in, name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp上传文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    
    public void delete(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            sftp.rm(name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp删除文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    
    private void mkdirs(ChannelSftp sftp, String dir) {
        String[] folders = dir.split("/");
        try {
            sftp.cd("/");
            for (String folder : folders) {
                if (folder.length() > 0) {
                    try {
                        sftp.cd(folder);
                    } catch (Exception e) {
                        sftp.mkdir(folder);
                        sftp.cd(folder);
                    }
                }
            }
        } catch (SftpException e) {
            throw new SftpPoolException("sftp创建目录出错", e);
        }
    }
}

Test

SftpTest.java

package com.vipsoft.sftp;

import com.vipsoft.sftp.utils.SftpUtil;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest
public class SftpTest {

    @Autowired
    private SftpUtil sftpUtil;

    @Test
    void downloadTest() {
        byte[] Dockerfiles = sftpUtil.download("/opt/demo/", "Dockerfile");
        System.out.println("FileSize =>" + dockerfiles.length);
    }

}

到此这篇关于Java GenericObjectPool 对象池化技术--SpringBoot sftp 连接池工具类的文章就介绍到这了,更多相关Java 对象池GenericObjectPool内容请搜索以前的文章或继续浏览下面的相关文章希望大家以后多多支持！

相关文章