tech.ml.dataset (TMD) is a Clojure library for tabular data processing similar to Python's Pandas, or R's data.table. It supports pragmatic data-intensive work on the JVM by providing powerful abstractions that simplify implementing efficient solutions to real problems. Datasets shrink in memory through columnar storage and the use of primitive arrays, packed datetime types, and string tables.
Unlike in Python or R, TMD datasets are functional, which means they're easier to reason about.
This fork adds the following functionality:
- Provide access to the stable sheet id. This id remains the same regardless of user input, ie rearrangement of sheets
- Use the fork ezand/fastexcel of the dhatim/fastexcel library to be able to access the stable id.
Installation instructions for your favorite build system (lein, deps.edn, etc...) can be found at Clojars, where the library is hosted:
- https://clojars.org/techascent/tech.ml.dataset
user> (require 'tech.v3.dataset)
nil
user> (->> (System/getProperties)
(map (fn [[k v]] {:k k :v (apply str (take 40 (str v)))}))
(tech.v3.dataset/->>dataset {:dataset-name "My Truncated System Properties"}))
My Truncated System Properties [53 2]:
| :k | :v |
|----------------------------|------------------------------------------|
| sun.desktop | gnome |
| awt.toolkit | sun.awt.X11.XToolkit |
| java.specification.version | 11 |
| sun.cpu.isalist | |
| sun.jnu.encoding | UTF-8 |
| java.class.path | src:resources:target/classes:/home/harol |
| java.vm.vendor | Ubuntu |
| sun.arch.data.model | 64 |
| java.vendor.url | https://ubuntu.com/ |
| user.timezone | America/Denver |
| ... | ... |
| os.arch | amd64 |
| java.vm.specification.name | Java Virtual Machine Specification |
| java.awt.printerjob | sun.print.PSPrinterJob |
| sun.os.patch.level | unknown |
| java.library.path | /usr/java/packages/lib:/usr/lib/x86_64-l |
| java.vm.info | mixed mode, sharing |
| java.vendor | Ubuntu |
| java.vm.version | 11.0.17+8-post-Ubuntu-1ubuntu222.04 |
| sun.io.unicode.encoding | UnicodeLittle |
| apple.awt.UIElement | true |
| java.class.version | 55.0 |The best place to start is the "Getting Started" topic in the documentation: https://techascent.github.io/tech.ml.dataset/000-getting-started.html
The "Walkthrough" topic provides long-form examples of processing real data: https://techascent.github.io/tech.ml.dataset/100-walkthrough.html
The "Quick Reference" topic summarizes many of the most frequently used functions: https://techascent.github.io/tech.ml.dataset/200-quick-reference.html
The API docs document every available function: https://techascent.github.io/tech.ml.dataset/
The provided Java API (javadoc / with frames) and sample program (source) show how to use TMD from Java.
- Log an issue!
- Visit the zulip stream.
- Or the slack data science channel.
- An alternative cutting-edge api with some important extra features is available via tablecloth.
- tech.v3.datatype provides the underlying numeric subsystem to TMD.
- Simple regression/classification machine learning pathways are available in tech.ml.
- Some independent benchmarks indicating TMD's speed.
- Bindings to a high performance in-process SQL database.
- A Graal native example project.
- The scicloj.ml tutorials may be a good way to jump straight into data science.
- Comparison between R's
data.table, R'sdplyr, and an older version of TMD. - Another overview of some of the available functions from genme: Some Functions
Copyright © 2023 Complements of TechAscent, LLC
Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.