做代加工的网站发布,住建部证书查询大厅,凡科免费建站,商城网站 没有什么文章 怎样优化Spark-core包设置默认的日志级别为info#xff0c;所以我们才看到一大堆日志信息。
开发的时候#xff0c;把print的日志掩盖了。 方法一#xff1a;
代码中设置日志级别 JavaSparkContext contextnew JavaSparkContext(conf);
context.setLogLevel(warn); 方…Spark-core包设置默认的日志级别为info所以我们才看到一大堆日志信息。
开发的时候把print的日志掩盖了。 方法一
代码中设置日志级别 JavaSparkContext contextnew JavaSparkContext(conf);
context.setLogLevel(warn); 方法二
添加 log4j.properties 到 resource log4j.properties #
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the License); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an AS IS BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
## Set everything to be logged to the console
log4j.rootCategoryWARN, console
log4j.appender.consoleorg.apache.log4j.ConsoleAppender
log4j.appender.console.targetSystem.err
log4j.appender.console.layoutorg.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n# Settings to quiet third party logs that are too verbose
log4j.logger.org.spark-project.jettyWARN
log4j.logger.org.spark-project.jetty.util.component.AbstractLifeCycleERROR
log4j.logger.org.apache.spark.repl.SparkIMain$exprTyperINFO
log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreterINFO
log4j.logger.org.apache.parquetERROR
log4j.logger.parquetERROR# SPARK-9183: Settings to avoid annoying messages when looking up nonexistent UDFs in SparkSQL with Hive support
log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandlerFATAL
log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistryERROR
