手机app日志分析系统（一）_as如何分析app日志

一、项目介绍
-------------------------------------------------
   1.App开发商
      每个开发商可以有多个App产品
 
   2.App软件
 
   3.数据服务平台提供商
      Umeng，
      向App开发商提供服务。提供App使用情况的统计服务。
 
   4.SDK
      数据服务平台提供商提供给App开发商的软件包。
      内置log上报程序。
 
   5.用户
      每个使用App的设备。
 
   6.租户
      购买了数据服务平台提供商服务的App开发商。
 
 
二、项目分析
------------------------------------------------------
   1.用户
      设备id,唯一性
 
   2.新增用户
      首次打开应用的用户。
      卸载再安装不是新增
 
   3.活跃用户
      指定时间段内打开过app的用户即为活跃用户。多次打开算一次。
 
   4.月活率
      活跃用户 / 截止到当月累计用户总数。
 
   5.沉默用户
      两天时间没有启动过app的用户就算沉默用户。
 
   6.版本分布
      计算各版本的新增用户、活跃用户、启动次数。
 
   7.本周回流用户
      上周没启动，本周启动的用户
 
   8.连续n周活跃用户
      连续n周，每周至少启动一次。
 
   9.忠诚用户
      连续5周以上活跃用户
 
   10.连续活跃用户
      连续2周以上
 
   11.近期流失用户
      连续n(2<= n <= 4)周没有启动应用的用户。
 
   12.留存用户
      某段时间内的新增用户，在经过一段时间后，仍然使用app的用户。
 
   13.用户新鲜度
      每天启动app的新老用户比例
 
   14.单次使用时长
      每次启动使用的时间长度。
 
   15.日使用时长
      每天的使用累加值。
 
   16.启动次数计算标准
      两次之间<30s,算作一次启动.
 
 
三、日志组成
--------------------------------------------------
    1.启动日志
 
    2.页面访问日志
 
    3.事件日志
 
    4.用户使用日志
 
    5.错误日志
 
 
四、开始项目 -- 初始化web日志收集程序
----------------------------------------------------
    1.创建新项目UmengProject
 
    2.创建公共模块app-analyze-common，存放用于跨模块间访问的类。添加maven依赖
        a.创建日志类AppLogEntity          

package com.test.app.common;
 
            /**
             * AppLog实体类
             * 内部含有各种日志时间的集合。
             */
            public class AppLogEntity {
                private String appId;                //应用唯一标识
                private String tenantId;             //租户唯一标识,企业用户
                private String deviceId;            //设备唯一标识
                private String appVersion;            //版本
                private String appChannel;            //渠道,安装时就在清单中制定了，appStore等。
                private String appPlatform;            //平台
                private String osType;                //操作系统
                private String deviceStyle;            //机型
 
                private AppStartupLog[] appStartupLogs;        //启动相关信息的数组
                private AppPageLog[] appPageLogs;            //页面跳转相关信息的数组
                private AppEventLog[] appEventLogs;            //事件相关信息的数组
                private AppUsageLog[] appUsageLogs;            //app使用情况相关信息的数组
                private AppErrorLog[] appErrorLogs;            //错误相关信息的数组
 
                /** get / set  **/
            }

        b.抽取共性，创建日志基本父类AppBaseLog
            

package com.test.app.common;
 
              import java.io.Serializable;
 
              /**
               * AppBaseLog
               */
              public class AppBaseLog implements Serializable {
                  private Long createdAtMs;            //日志创建时间
                  private String appId;                //应用唯一标识
                  private String tenantId;             //租户唯一标识,企业用户
                  private String deviceId;            //设备唯一标识
                  private String appVersion;            //版本
                  private String appChannel;            //渠道,安装时就在清单中制定了，appStore等。
                  private String appPlatform;            //平台
                  private String osType;                //操作系统
                  private String deviceStyle;            //机型
 
                  /*省略get set*/
              }

        c.创建日志具体分类的相关类AppStartupLog等
            ===================================================
            

package com.test.app.common;
 
            /**
             * 启动日志
             */
            public class AppStartupLog extends AppBaseLog {
                private String country;                 //国家，终端不用上报，服务器自动填充该属性
                private String province;                //省份，终端不用上报，服务器自动填充该属性
                private String ipAddress;               //ip地址
 
                private String network;                //网络
                private String carrier;                //运营商
 
                private String brand;               //品牌
                private String deviceStyle;            //机型
                private String screenSize;            //分辨率
                private String osType;                //操作系统
 
                //省略getset
 
            }

            ====================================================================
            

package com.test.app.common;
 
            /**
             * 应用上报的app错误日志相关信息
             */
            public class AppErrorLog extends AppBaseLog {
 
                private static final long serialVersionUID = 1L;
 
                private String errorBrief;    //错误摘要
                private String errorDetail;       //错误详情
            }

            =======================================================================
 
            

package com.test.app.common;
 
            import java.util.Map;
 
            /**
             * 应用上报的事件相关信息
             *
             */
            public class AppEventLog extends AppBaseLog {
 
                private static final long serialVersionUID = 1L;
 
                private String eventId;          //事件唯一标识
                private Long eventDurationSecs;    //事件持续时长
                private Map<String,String> paramKeyValueMap;      //参数名/值对
            }

            =========================================================================
           

 package com.test.app.common;
 
            /**
             * 应用上报的页面相关信息
             *
             */
            public class AppPageLog extends AppBaseLog {
 
                private static final long serialVersionUID = 1L;
 
                /*
                 * 一次启动中的页面访问次数(应保证每次启动的所有页面日志在一次上报中，即最后一条上报的页面记录的nextPage为空)
                 */
                private int pageViewCntInSession = 0;
 
                private String pageId;      //页面id
                private int visitIndex = 0;    //访问顺序号，0为第一个页面
                private String nextPage;   //下一个访问页面，如为空则表示为退出应用的页面
                private Long stayDurationSecs = (long) 0;  //当前页面停留时长
            }

            =============================================================================
           

 package com.test.app.common;
 
            /**
             * 应用上报的使用时长相关信息
             */
            public class AppUsageLog extends AppBaseLog {
 
                private static final long serialVersionUID = 1L;
 
                private Long singleUseDurationSecs;    //单次使用时长(秒数),指一次启动内应用在前台的持续时长
                private Long singleUploadTraffic;     //单次使用过程中的上传流量
                private Long singleDownloadTraffic;       //单次使用过程中的下载流量
            }

        d.创建util工具包com.test.app.util和属性拷贝工具类PropertiesUtil
            

package com.test.app.util;
 
            import java.beans.BeanInfo;
            import java.beans.IntrospectionException;
            import java.beans.Introspector;
            import java.beans.PropertyDescriptor;
            import java.lang.reflect.Method;
 
            /**
             * 通过内省实现属性复制
             */
            public class PropertiesUtil {
 
                //复制属性
                //要将AppLogEntity中的所有相关属性的值赋值给对用的具体log的属性值[因为具体log的属性值是空的]
                public static void copyProperties(Object src, Object des)
                {
                    try {
                        BeanInfo srcInfo = Introspector.getBeanInfo(src.getClass());
                        //属性描述符
                        PropertyDescriptor[] sarr = srcInfo.getPropertyDescriptors();
                        for(PropertyDescriptor p : sarr)
                        {
                            Method getter = p.getReadMethod();
                            Method setter = p.getWriteMethod();
                            //获取set方法描述符的name
                            String setName = setter.getName();
                            //获取set方法的参数类型
                            Class [] param = setter.getParameterTypes();
                            try {
                                //通过get方法描述符，获取src的属性值
                                Object value = getter.invoke(src);
                                //通过src set方法描述符找到des的set方法，给des的属性赋值
                                Method desSettrer = des.getClass().getMethod(setName, param);
                                desSettrer.invoke(des, value);
 
                            } catch (Exception e) {
                                //出现异常说明des中没有src的属性
                                continue;
                            }
                        }
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            }

    3.创建app日志收集web模块app-log-collect-web
        a.添加maven依赖
            <?xml version="1.0" encoding="UTF-8"?>
            <project xmlns="http://maven.apache.org/POM/4.0.0"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
                <modelVersion>4.0.0</modelVersion>
 
                <groupId>com.test</groupId>
                <artifactId>app-logs-collect-web</artifactId>
                <version>1.0-SNAPSHOT</version>
 
                <dependencies>
 
                    <dependency>
                        <groupId>junit</groupId>
                        <artifactId>junit</artifactId>
                        <version>4.11</version>
                    </dependency>
                    <dependency>
                        <groupId>com.fasterxml.jackson.core</groupId>
                        <artifactId>jackson-core</artifactId>
                        <version>2.8.8</version>
                    </dependency>
                    <dependency>
                        <groupId>com.fasterxml.jackson.core</groupId>
                        <artifactId>jackson-databind</artifactId>
                        <version>2.8.3</version>
                    </dependency>
 
                    <dependency>
                        <groupId>com.maxmind.db</groupId>
                        <artifactId>maxmind-db</artifactId>
                        <version>1.0.0</version>
                    </dependency>
 
                    <dependency>
                        <groupId>org.springframework</groupId>
                        <artifactId>spring-webmvc</artifactId>
                        <version>4.3.5.RELEASE</version>
                    </dependency>
                    <dependency>
                        <groupId>javax.servlet</groupId>
                        <artifactId>servlet-api</artifactId>
                        <version>2.5</version>
                    </dependency>
                    <dependency>
                        <groupId>com.test</groupId>
                        <artifactId>app-analyze-common</artifactId>
                        <version>1.0-SNAPSHOT</version>
                    </dependency>
                    <dependency>
                        <groupId>com.alibaba</groupId>
                        <artifactId>fastjson</artifactId>
                        <version>1.2.24</version>
                    </dependency>
                </dependencies>
            </project>
 
        b.添加app-analyze-common模块依赖，共享模块资源
            Project-Structure --> dependencies --> 3.Module De...
            别忘记打钩，不然引用不了
 
        c.创建java包
            com.test.applogs.collect.web.controller
 
        d.编写WEB-INF/web.xml
            <?xml version="1.0" encoding="UTF-8"?>
            <web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
                     version="3.1">
                <servlet>
                    <servlet-name>controller</servlet-name>
                    <servlet-class>org.springframework.web.servlet.DispatcherServlet
                    </servlet-class>
                </servlet>
                <servlet-mapping>
                    <servlet-name>controller</servlet-name>
                    <url-pattern>/</url-pattern>
                </servlet-mapping>
 
            </web-app>
 
        e.创建新文件WEB-INF/controller-servlet.xml
            <?xml version="1.0" encoding="UTF-8"?>
            <beans xmlns="http://www.springframework.org/schema/beans"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xmlns:mvc="http://www.springframework.org/schema/mvc"
                   xmlns:context="http://www.springframework.org/schema/context"
                   xsi:schemaLocation="http://www.springframework.org/schema/beans
                                    http://www.springframework.org/schema/beans/spring-beans.xsd
                                    http://www.springframework.org/schema/mvc
                                    http://www.springframework.org/schema/mvc/spring-mvc-4.3.xsd
                                    http://www.springframework.org/schema/context
                                    http://www.springframework.org/schema/context/spring-context-4.3.xsd">
                <!-- 配置扫描路径 -->
                <context:component-scan
                        base-package="com.test.applogs.collect.web.controller"/>
                <!-- 使用注解驱动 -->
                <mvc:annotation-driven/>
                <!-- 内部资源视图解析器 -->
                <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
                    <property name="prefix" value="/views"/>
                    <property name="suffix" value=".jsp"/>
                </bean>
                <!-- 此处乃进行json数据传输的关键,当配置 -->
                <bean id="jsonMapping"  class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter"/>
                <bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter">
                    <property name="messageConverters">
                        <list>
                            <ref bean="jsonMapping"/>
                        </list>
                    </property>
                </bean>
            </beans>
 
        f.创建日志收集控制器
            

package com.test.applogs.collect.web.controller;
 
            import com.alibaba.fastjson.JSONObject;
            import com.test.app.util.PropertiesUtil;
            import com.test.app.common.*;
            import com.test.app.common.AppLogEntity;
            import com.test.app.common.AppStartupLog;
            import org.springframework.stereotype.Controller;
            import org.springframework.web.bind.annotation.RequestBody;
            import org.springframework.web.bind.annotation.RequestMapping;
            import org.springframework.web.bind.annotation.RequestMethod;
            import org.springframework.web.bind.annotation.ResponseBody;
 
            import javax.servlet.http.HttpServletRequest;
 
            /**
             */
            @Controller()
            @RequestMapping("/coll")
            public class CollectLogController {
 
                /**
                 * 启动日志收集
                 */
                @RequestMapping(value = "/index", method = RequestMethod.POST)
                @ResponseBody
                public AppLogEntity collect(@RequestBody AppLogEntity e, HttpServletRequest req) {
 
                    System.out.println("=============================");
                    //server时间
                    long myTime = System.currentTimeMillis() ;
                    //客户端时间
                    long clientTime = Long.parseLong(req.getHeader("clientTime"));
                    //时间校对
                    long diff = myTime - clientTime ;
 
                    //对e进行处理，将具体日志分类的属性值填充完毕
                    processLogs(e);
                    //修正日志时间
                    verifyTime(e,diff);
                    String json = JSONObject.toJSONString(e);
 
                    System.out.println(json);
                    return e;
                }
 
                /**
                 * 校对各个具体日志的创建时间（使用服务器时间差diff）
                 */
                private void verifyTime(AppLogEntity e, long diff)
                {
                    //启动修正
                    //startuplog
                    for(AppBaseLog log : e.getAppStartupLogs()){
                        log.setCreatedAtMs(log.getCreatedAtMs() + diff );
                    }
                    for(AppBaseLog log : e.getAppUsageLogs()){
                        log.setCreatedAtMs(log.getCreatedAtMs() + diff );
                    }
                    for(AppBaseLog log : e.getAppPageLogs()){
                        log.setCreatedAtMs(log.getCreatedAtMs() + diff );
                    }
                    for(AppBaseLog log : e.getAppEventLogs()){
                        log.setCreatedAtMs(log.getCreatedAtMs() + diff );
                    }
                    for(AppBaseLog log : e.getAppErrorLogs()){
                        log.setCreatedAtMs(log.getCreatedAtMs() + diff );
                    }
                }
 
                /**
                 * 将Log的属性分类复制到各个具体的log中
                 */
                private void processLogs(AppLogEntity e){
 
                    for(AppStartupLog log : e.getAppStartupLogs()){
                        PropertiesUtil.copyProperties(e,log);
                    }
                    for(AppErrorLog log : e.getAppErrorLogs()){
                        PropertiesUtil.copyProperties(e,log);
                    }
                    for(AppEventLog log : e.getAppEventLogs()){
                        PropertiesUtil.copyProperties(e,log);
                    }
                    for(AppPageLog log : e.getAppPageLogs()){
                        PropertiesUtil.copyProperties(e,log);
                    }
                    for(AppUsageLog log : e.getAppUsageLogs()){
                        PropertiesUtil.copyProperties(e,log);
                    }
                }
            }

        g.创建tomcat-server启动程序
            Run --> Edit Configurations --> + --> tomcat server --> ...
            Project Structure --> Artifacts --> Add All Maven Jars --> ...
            [注意！]Project Structure --> Artifacts --> Add Common模块到out class --> 不然依赖的其他模块是加载不到的，同时添加maven依赖的支持
 
 
    4.创建app-logs-phone模块，用于模拟手机生成日志
        a.添加maven依赖[可以通过maven的手段，达到访问其他模块的内容，比如common模块]
            <?xml version="1.0" encoding="UTF-8"?>
            <project xmlns="http://maven.apache.org/POM/4.0.0"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
                <modelVersion>4.0.0</modelVersion>
 
                <groupId>com.test</groupId>
                <artifactId>app-logs-phone</artifactId>
                <version>1.0-SNAPSHOT</version>
 
                <dependencies>
                    <dependency>
                        <groupId>com.alibaba</groupId>
                        <artifactId>fastjson</artifactId>
                        <version>1.2.24</version>
                    </dependency>
                    <dependency>
                        <groupId>com.test</groupId>
                        <artifactId>app-analyze-common</artifactId>
                        <version>1.0-SNAPSHOT</version>
                    </dependency>
                </dependencies>
            </project>
 
        b.创建海量日志生成模拟程序
            

package com.test.app.client;
 
            import com.alibaba.fastjson.JSONObject;
            import com.test.app.common.*;
            //import com.test.app.util.PropertiesUtil;
 
            import java.io.File;
            import java.io.FileWriter;
            import java.io.InputStream;
            import java.net.HttpURLConnection;
            import java.net.URL;
            import java.util.HashMap;
            import java.util.Map;
            import java.util.Random;
 
            /**
             * 数据生成程序
             */
            public class TestGenData {
 
               /**
                *
                */
               private static String url = "http://localhost:8080/coll/index";
 
               private static Random random = new Random();
 
               private static String appId = "sdk34734";
               private static String[] tenantIds = {"cake"};
               private static String[] deviceIds = initDeviceId();
               private static String[] appVersions = {"3.2.1", "3.2.2"};
               private static String[] appChannels = {"youmeng1", "youmeng2"};
               private static String[] appPlatforms = {"android", "ios"};
 
               private static Long[] createdAtMsS = initCreatedAtMs();
               //国家，终端不用上报，服务器自动填充该属性
               private static String[] countrys = {"America", "china"};
               //省份，终端不用上报，服务器自动填充该属性
               private static String[] provinces = {"Washington", "jiangxi", "beijing"};
               //网络
               private static String[] networks = {"WiFi", "CellNetwork"};
               //运营商
               private static String[] carriers = {"中国移动", "中国电信", "EE"};
               //机型
               private static String[] deviceStyles = {"iPhone 6", "iPhone 6 Plus", "红米手机1s"};
               //分辨率
               private static String[] screenSizes = {"1136*640", "960*640", "480*320"};
               //操作系统
               private static String[] osTypes = {"8.3", "7.1.1"};
               //品牌
               private static String[] brands = {"三星", "华为", "Apple", "魅族", "小米", "锤子"};
               //事件唯一标识
               private static String[] eventIds = {"popMenu", "autoImport", "BookStore"};
               //事件持续时长
               private static Long[] eventDurationSecsS = {new Long(25), new Long(67), new Long(45)};
 
               static Map<String, String> map1 = new HashMap<String, String>() {
                  {
                     put("testparam1key", "testparam1value");
                     put("testparam2key", "testparam2value");
                  }
               };
               static Map<String, String> map2 = new HashMap<String, String>() {
                  {
                     put("testparam3key", "testparam3value");
                     put("testparam4key", "testparam4value");
                  }
               };
               private static Map[] paramKeyValueMapsS = {map1, map2};
 
               //单次使用时长(秒数),指一次启动内应用在前台的持续时长
               private static Long[] singleUseDurationSecsS = initSingleUseDurationSecs();
 
               private static String[] errorBriefs = {"at cn.lift.dfdf.web.AbstractBaseController.validInbound(AbstractBaseController.java:72)", "at cn.lift.appIn.control.CommandUtil.getInfo(CommandUtil.java:67)"};        //错误摘要
               private static String[] errorDetails = {"java.lang.NullPointerException\\n    " + "at cn.lift.appIn.web.AbstractBaseController.validInbound(AbstractBaseController.java:72)\\n " + "at cn.lift.dfdf.web.AbstractBaseController.validInbound", "at cn.lift.dfdfdf.control.CommandUtil.getInfo(CommandUtil.java:67)\\n " + "at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\\n" + " at java.lang.reflect.Method.invoke(Method.java:606)\\n"};        //错误详情
               //页面id
               private static String[] pageIds = {"list.html", "main.html", "test.html"};
               //访问顺序号，0为第一个页面
               private static int[] visitIndexs = {0, 1, 2, 3, 4};
               //下一个访问页面，如为空则表示为退出应用的页面
               private static String[] nextPages = {"list.html", "main.html", "test.html", null};
               //当前页面停留时长
               private static Long[] stayDurationSecsS = {new Long(45), new Long(2), new Long(78)};
 
               //启动相关信息的数组
               private static AppStartupLog[] appStartupLogs = initAppStartupLogs();
               //页面跳转相关信息的数组
               private static AppPageLog[] appPageLogs = initAppPageLogs();
               //事件相关信息的数组
               private static AppEventLog[] appEventLogs = initAppEventLogs();
               //app使用情况相关信息的数组
               private static AppUsageLog[] appUsageLogs = initAppUsageLogs();
               //错误相关信息的数组
               private static AppErrorLog[] appErrorLogs = initAppErrorLogs();
 
               private static String[] initDeviceId() {
                  String base = "device22";
                  String[] result = new String[100];
                  for (int i = 0; i < 100; i++) {
                     result[i] = base + i + "";
                  }
                  return result;
               }
 
               private static Long[] initCreatedAtMs() {
                  Long createdAtMs = System.currentTimeMillis();
                  Long[] result = new Long[11];
                  for (int i = 0; i < 10; i++) {
                     result[i] = createdAtMs - (long) (i * 24 * 3600 * 1000);
                  }
                  result[10] = createdAtMs;
                  return result;
               }
 
               private static Long[] initSingleUseDurationSecs() {
                  Random random = new Random();
                  Long[] result = new Long[200];
                  for (int i = 1; i < 200; i++) {
                     result[i] = (long) random.nextInt(200);
                  }
                  return result;
               }
 
               //启动相关信息的数组
               private static AppStartupLog[] initAppStartupLogs() {
                  AppStartupLog[] result = new AppStartupLog[10];
                  for (int i = 0; i < 10; i++) {
                     AppStartupLog appStartupLog = new AppStartupLog();
                     appStartupLog.setCountry(countrys[random.nextInt(countrys.length)]);
                     appStartupLog.setProvince(provinces[random.nextInt(provinces.length)]);
                     appStartupLog.setNetwork(networks[random.nextInt(networks.length)]);
                     appStartupLog.setCarrier(carriers[random.nextInt(carriers.length)]);
                     appStartupLog.setDeviceStyle(deviceStyles[random.nextInt(deviceStyles.length)]);
                     appStartupLog.setScreenSize(screenSizes[random.nextInt(screenSizes.length)]);
                     appStartupLog.setOsType(osTypes[random.nextInt(osTypes.length)]);
                     appStartupLog.setBrand(brands[random.nextInt(brands.length)]);
                     appStartupLog.setCreatedAtMs(createdAtMsS[random.nextInt(createdAtMsS.length)]);
                     result[i] = appStartupLog;
                  }
                  return result;
               }
 
               //页面跳转相关信息的数组
               private static AppPageLog[] initAppPageLogs() {
                  AppPageLog[] result = new AppPageLog[10];
                  for (int i = 0; i < 10; i++) {
                     AppPageLog appPageLog = new AppPageLog();
                     String pageId = pageIds[random.nextInt(pageIds.length)];
                     int visitIndex = visitIndexs[random.nextInt(visitIndexs.length)];
                     String nextPage = nextPages[random.nextInt(nextPages.length)];
                     while (pageId.equals(nextPage)) {
                        nextPage = nextPages[random.nextInt(nextPages.length)];
                     }
                     Long stayDurationSecs = stayDurationSecsS[random.nextInt(stayDurationSecsS.length)];
 
                     appPageLog.setPageId(pageId);
                     appPageLog.setStayDurationSecs(stayDurationSecs);
                     appPageLog.setVisitIndex(visitIndex);
                     appPageLog.setNextPage(nextPage);
                     appPageLog.setCreatedAtMs(createdAtMsS[random.nextInt(createdAtMsS.length)]);
                     result[i] = appPageLog;
                  }
                  return result;
               }
 
               ;
 
               //事件相关信息的数组
               private static AppEventLog[] initAppEventLogs() {
                  AppEventLog[] result = new AppEventLog[10];
                  for (int i = 0; i < 10; i++) {
                     AppEventLog appEventLog = new AppEventLog();
                     appEventLog.setEventId(eventIds[random.nextInt(eventIds.length)]);
                     appEventLog.setParamKeyValueMap(paramKeyValueMapsS[random.nextInt(paramKeyValueMapsS.length)]);
                     appEventLog.setEventDurationSecs(eventDurationSecsS[random.nextInt(eventDurationSecsS.length)]);
                     appEventLog.setCreatedAtMs(createdAtMsS[random.nextInt(createdAtMsS.length)]);
                     result[i] = appEventLog;
                  }
                  return result;
               }
 
               ;
 
               //app使用情况相关信息的数组
               private static AppUsageLog[] initAppUsageLogs() {
                  AppUsageLog[] result = new AppUsageLog[10];
                  for (int i = 0; i < 10; i++) {
                     AppUsageLog appUsageLog = new AppUsageLog();
                     appUsageLog.setSingleUseDurationSecs(singleUseDurationSecsS[random.nextInt(singleUseDurationSecsS.length)]);
                     appUsageLog.setCreatedAtMs(createdAtMsS[random.nextInt(createdAtMsS.length)]);
                     result[i] = appUsageLog;
                  }
                  return result;
               }
 
               ;
 
               //错误相关信息的数组
               private static AppErrorLog[] initAppErrorLogs() {
                  AppErrorLog[] result = new AppErrorLog[10];
                  for (int i = 0; i < 10; i++) {
                     AppErrorLog appErrorLog = new AppErrorLog();
                     appErrorLog.setErrorBrief(errorBriefs[random.nextInt(errorBriefs.length)]);
                     appErrorLog.setErrorDetail(errorDetails[random.nextInt(errorDetails.length)]);
                     appErrorLog.setCreatedAtMs(createdAtMsS[random.nextInt(createdAtMsS.length)]);
                     appErrorLog.setOsType(osTypes[random.nextInt(osTypes.length)]);
                     appErrorLog.setDeviceStyle(deviceStyles[random.nextInt(deviceStyles.length)]);
                     result[i] = appErrorLog;
                  }
                  return result;
               }
 
               private static void httpPost(String urlString, String params) {
                  URL url;
 
                  try {
                     url = new URL(urlString);
                     HttpURLConnection conn = (HttpURLConnection) url.openConnection();
                     conn.setRequestMethod("POST");
                     conn.setDoOutput(true);
                     conn.setDoInput(true);
                     conn.setUseCaches(false);
                     conn.setInstanceFollowRedirects(true);
                     conn.setRequestProperty("User-Agent",
                           "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0");
                     conn.setRequestProperty("Content-Type", "application/json");
                     conn.setConnectTimeout(1000 * 5);
                     conn.connect();
                     conn.getOutputStream().write(params.getBytes("utf8"));
                     conn.getOutputStream().flush();
                     conn.getOutputStream().close();
                     byte[] buffer = new byte[1024];
                     StringBuffer sb = new StringBuffer();
                     InputStream in = conn.getInputStream();
                     int httpCode = conn.getResponseCode();
                     System.out.println(in.available());
                     while (in.read(buffer, 0, 1024) != -1) {
                        sb.append(new String(buffer));
                     }
                     System.out.println("sb:" + sb.toString());
                     in.close();
                     System.out.println(httpCode);
                  } catch (Exception e) {
                     e.printStackTrace();
                  }
               }
 
               public static void main(String[] args) {
                  Test1();
               }
 
               private static void Test1() {
                  Random random = new Random();
                  try {
                     //发送数据
                     for (int i = 1; i <= 2000; i++) {
                        AppLogEntity logEntity = new AppLogEntity();
                        //渠道
                        logEntity.setAppChannel(appChannels[random.nextInt(appChannels.length)]);
                        //appid
                        logEntity.setAppId(appId);
                        //platform
                        logEntity.setAppPlatform(appPlatforms[random.nextInt(appPlatforms.length)]);
                        logEntity.setAppVersion(appVersions[random.nextInt(appVersions.length)]);
                        String tenantId = tenantIds[random.nextInt(tenantIds.length)];
                        if (tenantId != null) {
                           logEntity.setTenantId(tenantId);
                        }
                        logEntity.setTenantId(tenantIds[random.nextInt(tenantIds.length)]);
                        logEntity.setDeviceId(deviceIds[random.nextInt(deviceIds.length)]);
 
                        //模拟startup log集合
                        logEntity.setAppStartupLogs(new AppStartupLog[]{appStartupLogs[random.nextInt(appStartupLogs.length)]});
                        logEntity.setAppEventLogs(new AppEventLog[]{appEventLogs[random.nextInt(appEventLogs.length)]});
                        logEntity.setAppErrorLogs(new AppErrorLog[]{appErrorLogs[random.nextInt(appErrorLogs.length)]});
                        logEntity.setAppPageLogs(new AppPageLog[]{appPageLogs[random.nextInt(appPageLogs.length)]});
                        logEntity.setAppUsageLogs(new AppUsageLog[]{appUsageLogs[random.nextInt(appUsageLogs.length)]});
                        try {
                           //将对象转换成json string
                           String json = JSONObject.toJSONString(logEntity);
                           UploadUtil.upload(json);
                           Thread.sleep(2000);
                        } catch (Exception ex) {
                           System.out.println(ex);
                        }
                     }
                  } catch (Exception ex) {
                     ex.printStackTrace();
                  }
               }
 
               private static void Test2() {
                  boolean result = map1.isEmpty();
                  System.out.println(result);
               }
            }

        c.创建UploadUtil类，模拟日志上传到web
           

package com.test.app.client;
 
           import java.io.InputStream;
           import java.io.OutputStream;
           import java.net.HttpURLConnection;
           import java.net.URL;
 
           /**
            * 模拟手机上报日志程序
            */
           public class UploadUtil {
 
               /**
                * 上传日志
                */
               public static void upload(String json) throws Exception {
                  try{
                     //输入流
                     InputStream in = ClassLoader.getSystemResourceAsStream("log.json");
 
                     URL url = new URL("http://localhost:8080/coll/index");
                     HttpURLConnection conn = (HttpURLConnection) url.openConnection();
                     //设置请求方式为post
                     conn.setRequestMethod("POST");
 
                     //时间头用来供server进行时钟校对的
                     conn.setRequestProperty("clientTime",System.currentTimeMillis() + "");
                     //允许上传数据
                     conn.setDoOutput(true);
                     //设置请求的头信息,设置内容类型
                     conn.setRequestProperty("Content-Type", "application/json");
 
 
                     //输出流
                     OutputStream out = conn.getOutputStream();
                     out.write(json.getBytes());
                     out.flush();
                     out.close();
                     in.close();
                     int code = conn.getResponseCode();
                     System.out.println(code);
                  }
                  catch (Exception e){
                     e.printStackTrace();
                  }
               }
           }

        d.生产环境下特别注意时钟问题
            手机客户端的时间可能不准，所以不能按照客户端提供的时间来算
            所以。服务器端收集文件之后要进行时间校对
                1)client
                    发送数据同时，写入clientTime头。
                    conn.setRequestProperty("clientTime",System.currentTimeMillis() + "");
                2)web server
                    //server时间
                    long myTime = System.currentTimeMillis() ;
                    //客户端时间
                    long clientTime = Long.parseLong(req.getHeader("clientTime"));
                    //时间校对
                    long diff = myTime - clientTime ;
 
                3)完成Log实体中公共部分属性和Log类中间属性复制。