Alien-XGBoost
view release on metacpan or search on metacpan
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
xgboost/R-package/tests/testthat/test_helpers.R view on Meta::CPAN
xgb.attributes(bst) <- list(a=NULL, b=NULL)
expect_equal(xgb.attributes(bst), list.default)
xgb.attributes(bst) <- list(niter=NULL)
expect_null(xgb.attributes(bst))
})
if (grepl('Windows', Sys.info()[['sysname']]) ||
grepl('Linux', Sys.info()[['sysname']]) ||
grepl('Darwin', Sys.info()[['sysname']])) {
test_that("xgb-attribute numeric precision", {
# check that lossless conversion works with 17 digits
# numeric -> character -> numeric
X <- 10^runif(100, -20, 20)
X2X <- as.numeric(format(X, digits = 17))
expect_identical(X, X2X)
# retrieved attributes to be the same as written
for (x in X) {
xgb.attr(bst.Tree, "x") <- x
expect_identical(as.numeric(xgb.attr(bst.Tree, "x")), x)
xgb.attributes(bst.Tree) <- list(a = "A", b = x)
expect_identical(as.numeric(xgb.attr(bst.Tree, "b")), x)
xgboost/dmlc-core/README.md view on Meta::CPAN
* Try to introduce minimum dependency when possible
### CheckList before submit code
* Type ```make lint``` and fix all the style problems.
* Type ```make doc``` and fix all the warnings.
NOTE
----
deps:
libcurl4-openssl-dev
xgboost/dmlc-core/cmake/Modules/FindCrypto.cmake view on Meta::CPAN
IF(CRYPTO_LIBRARIES)
SET(Crypto_FIND_QUIETLY TRUE)
ENDIF(CRYPTO_LIBRARIES)
IF(SSL_EAY_DEBUG AND SSL_EAY_RELEASE)
SET(LIB_FOUND 1)
ENDIF(SSL_EAY_DEBUG AND SSL_EAY_RELEASE)
FIND_PATH(CRYPTO_INCLUDE_DIR openssl/crypto.h )
FIND_LIBRARY(CRYPTO_LIBRARIES NAMES crypto )
IF(CRYPTO_INCLUDE_DIR AND CRYPTO_LIBRARIES)
SET(CRYPTO_FOUND TRUE)
ELSE(CRYPTO_INCLUDE_DIR AND CRYPTO_LIBRARIES)
SET(CRYPTO_FOUND FALSE)
ENDIF (CRYPTO_INCLUDE_DIR AND CRYPTO_LIBRARIES)
IF (CRYPTO_FOUND)
IF (NOT Crypto_FIND_QUIETLY)
xgboost/dmlc-core/make/dmlc.mk view on Meta::CPAN
DMLC_LDFLAGS+= $(HDFS_LIB_PATH)/libhdfs.a
endif
DMLC_LDFLAGS += -L$(LIBJVM) -ljvm -Wl,-rpath=$(LIBJVM)
else
DMLC_CFLAGS+= -DDMLC_USE_HDFS=0
endif
# setup S3
ifeq ($(USE_S3),1)
DMLC_CFLAGS+= -DDMLC_USE_S3=1
DMLC_LDFLAGS+= -lcurl -lssl -lcrypto
else
DMLC_CFLAGS+= -DDMLC_USE_S3=0
endif
ifeq ($(USE_GLOG), 1)
DMLC_CFLAGS += -DDMLC_USE_GLOG=1
DMLC_LDFLAGS += -lglog
endif
ifeq ($(USE_AZURE),1)
xgboost/dmlc-core/src/io/s3_filesys.cc view on Meta::CPAN
// Copyright by Contributors
extern "C" {
#include <errno.h>
#include <curl/curl.h>
#include <curl/curl.h>
#include <openssl/hmac.h>
#include <openssl/md5.h>
#include <openssl/bio.h>
#include <openssl/buffer.h>
}
#include <dmlc/io.h>
#include <dmlc/logging.h>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <string>
#include <algorithm>
#include <ctime>
#include <sstream>
xgboost/doc/jvm/xgboost4j-intro.md view on Meta::CPAN
## System Overview
In the following Figure, we describe the overall architecture of XGBoost4J. XGBoost4J provides the Java/Scala API calling the core functionality of XGBoost library. Most importantly, it not only supports the single-machine model training, but also pr...
By calling the XGBoost4J API, users can scale the model training to the cluster. XGBoost4J calls the running instance of XGBoost worker in Spark/Flink task and run them across the cluster. The communication among the distributed model training tasks ...
With the abstraction of XGBoost4J, users can build an unified data analytic application ranging from Extract-Transform-Loading, data exploration, machine learning model training and the final data product service. The following figure illustrate an e...
## Single-machine Training Walk-through
In this section, we will work through the APIs of XGBoost4J by examples.
We will be using scala for demonstration, but we also have a complete API for java users.
To start the model training and evaluation, we need to prepare the training and test set:
xgboost/doc/jvm/xgboost4j_full_integration.md view on Meta::CPAN
In the last months, we have a lot of communication with the users and gain the deeper understanding of the users' latest usage scenario and requirements:
* XGBoost keeps gaining more and more deployments in the production environment and the adoption in machine learning competitions [Link](http://datascience.la/xgboost-workshop-and-meetup-talk-with-tianqi-chen/).
* While Spark is still the mainstream data processing tool in most of scenarios, more and more users are porting their RDD-based Spark programs to [DataFrame/Dataset APIs](http://spark.apache.org/docs/latest/sql-programming-guide.html) for the well-d...
* Spark itself has presented a clear roadmap that DataFrame/Dataset would be the base of the latest and future features, e.g. latest version of [ML pipeline](http://spark.apache.org/docs/latest/ml-guide.html) and [Structured Streaming](http://spark.a...
Based on these feedbacks from the users, we observe a gap between the original RDD-based XGBoost4J-Spark and the users' latest usage scenario as well as the future direction of Spark ecosystem. To fill this gap, we start working on the <b><i>integrat...
## A Full Integration of XGBoost and DataFrame/Dataset
The following figure illustrates the new pipeline architecture with the latest XGBoost4J-Spark.
Being different with the previous version, users are able to use both low- and high-level memory abstraction in Spark, i.e. RDD and DataFrame/Dataset. The DataFrame/Dataset abstraction grants the user to manipulate structured datasets and utilize the...
xgboost/doc/jvm/xgboost4j_full_integration.md view on Meta::CPAN
// drop the extra column
val indexed = indexer.fit(salesDF).transform(df).drop("storeType")
// use the transformed dataframe as training dataset
val xgboostModel = XGBoost.trainWithDataFrame(
indexed, paramMap, numRound, nWorkers, useExternalMemory)
```
#### Pipelining
Spark ML package allows the user to build a complete pipeline from feature extraction/transformation/selection to model training. We integrate XGBoost with ML package and make it feasible to embed XGBoost into such a pipeline seamlessly. The followin...
```scala
import org.apache.spark.ml.feature.StringIndexer
// load sales records saved in json files
val salesDF = spark.read.json("sales.json")
// transform the string-represented storeType feature to numeric storeTypeIndex
val indexer = new StringIndexer()
.setInputCol("storeType")
xgboost/doc/jvm/xgboost4j_full_integration.md view on Meta::CPAN
.setEstimator(xgbEstimator)
.setEvaluator(new RegressionEvaluator().setLabelCol("sales"))
.setEstimatorParamMaps(paramGrid)
.setTrainRatio(0.8)
val salesTestDF = spark.read.json("sales_test.json")
val salesRecordsWithPred = xgboostModel.transform(salesTestDF)
```
## Summary
Through the latest XGBoost4J-Spark, XGBoost users can build a more efficient data processing pipeline which works with DataFrame/Dataset APIs to handle the structured data with the excellent performance, and simultaneously embrace the powerful XGBoos...
The latest version of XGBoost4J-Spark has been available in the [GitHub Repository] (https://github.com/dmlc/xgboost), and the latest API docs are in [here](http://xgboost.readthedocs.io/en/latest/jvm/index.html).
## Portable Machine Learning Systems
XGBoost is one of the projects incubated by [Distributed Machine Learning Community (DMLC)](http://dmlc.ml/), which also creates several other popular projects on machine learning systems ([Link](https://github.com/dmlc/)), e.g. one of the most popul...
## Further Readings
( run in 2.246 seconds using v1.01-cache-2.11-cpan-acf6aa7dc9e )