kafka消费者和生产者为内/外网映射情况的配置_kafka cdc 配置字段映射

IP：  (请修改自己的ip地址)

    内网：172.18.10.10          

    外网：120.78.22.22

1. 先在/etc/hosts文件中添加解析记录
            172.18.10.10    HostName
                
2. 修改 kafka_2.11-2.0.0/config    server.properties
        listeners=PLAINTEXT://0.0.0.0:9092
        advertised.listeners=PLAINTEXT://120.78.22.22:9092
        advertised.host.name=HostName  

有用户密码验证的情况下

IP：  (请修改自己的ip地址)

    内网：172.18.10.10          

    外网：120.78.22.22

1. 先在/etc/hosts文件中添加解析记录
            172.18.10.10    HostName
                
2. 修改 kafka_2.11-2.0.0/config    server.properties
        listeners=SASL_PLAINTEXT:://0.0.0.0:9092
        advertised.listeners=SASL_PLAINTEXT:://120.78.22.22:9092
        advertised.host.name=HostName    

代用户密码验证的java客户端连接

 Properties props = new Properties();
        Map map = new Gson().fromJson(kafkaServer,Map.class);
        //集群地址，多个地址用"，"分隔
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,map.get("bootstrap.servers"));
        //设置消费者的group id
        props.put("group.id", taskid+"");
        props.put(SaslConfigs.SASL_MECHANISM, "PLAIN");
        props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SASL_PLAINTEXT");
//        props.put(SaslConfigs.SASL_JAAS_CONFIG, "org.apache.kafka.common.security.plain.PlainLoginModule \
//                "username="admin\" password=\"<api-secret>\";");
        props.put(SaslConfigs.SASL_JAAS_CONFIG,"org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"chinaums@000000\";");
        //如果为真，consumer所消费消息的offset将会自动的同步到zookeeper。如果消费者死掉时，由新的consumer使用继续接替
        props.put("enable.auto.commit", "false");
        props.put("max.poll.records",3000);
        //consumer向zookeeper提交offset的频率
//        props.put("auto.commit.interval.ms", "1000");
        props.put("session.timeout.ms", "30000");
        props.put("auto.offset.reset","earliest");
        props.put("fetch.max.bytes",524288000);
        //反序列化
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        //创建消费者
        KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
//        TopicPartition partition0 = new TopicPartition(topic, 0);
        // 订阅topic，可以为多个用,隔开，此处订阅了"test-partition-1", "test"这两个主题
        consumer.subscribe(Arrays.asList(topic));