Automatically add transactions from all major Israeli banks and credit card companies to a online worksheet

Internally we use israeli-bank-scrapers to scrape the data.

Having all your data in one place lets you view all of your expenses in a beautiful dashboard like Google Data Studio, Azure Data Explorer dashboards, Microsoft Power BI and YNAB.

This app requires some technical skills, if you prefer a GUI app you can use Caspion instead.

Important: The current implementation assumes that you run the code on secure and trusted computers.

It’s a bad idea to put all your financial data and passwords in one place, especially with more than read-only access.

By using moneyman, you acknowledge that you are taking full responsibility for the code quality and will use it only after you review the code and validate that it’s secure.

Please use a proper secret management solution to save and pass the environment variables

Moneyman can be configured to periodically run automatically, using the scrape github workflow.

By default, this workflow will run every other day.

Since logs are public for public repos, most logs are off by default and the progress and error messages will be sent in telegram.

1. Fork the moneyman repo to your account
2. Add the following secrets to the actions secrets of the forked repo
   1. ACCOUNTS_JSON - So moneyman can login to your accounts
   2. TELEGRAM_API_[KEY, CHAT_ID] - So moneyman can send private logs and errors
   3. The environment variables of the storage you want to use
3. Build and upload the docker image using the "Run workflow" button in workflows/build.yml
4. Wait for the scrape workflow to be triggered by github

1. Clone this repo
2. Run npm install
3. Run npm run build
4. Add your env variables (you can add them in a .env file in the project's root directory)
5. Run npm run start

1. Define the environment variables in a .env file
2. docker run --rm --env-file ".env" ghcr.io/daniel-hauser/moneyman:latest.

docker doesn't support multiline environment variables (i.e. GOOGLE_SERVICE_ACCOUNT_PRIVATE_KEY), in that case you can run docker-compose up instead

We use the debug package for debug messages under the moneyman: namespace.

If you want to see them, use the DEBUG environment variable with the value moneyman:*

Use the following env vars to setup the data fetching:

A json array of accounts following this schema with an additional companyId field with a companyType as the value.

Example:

[
  { "companyId": "hapoalim", "userCode": "AB1234", "password": "p@ssword" },
  { "companyId": "visaCal", "username": "Ploni Almoni", "password": "p@ssword" }
]

We use telegram to send you the update status.

1. Create your bot following this
2. Open this url https://api.telegram.org/bot<TELEGRAM_API_KEY>/getUpdates
3. Send a message to your bot and fnd the chat id

Use the following env vars to setup:

TODO: Add a way to send a message to the bot to connect?

1. Create a new data explorer cluster (can be done for free here)

2. Create a database within your cluster

3. Create a azure Service Principal following steps 1-7 here

4. Allow the service to ingest data to the database by running this:

   .execute database script <|
   .add database ['<ADE_DATABASE_NAME>'] ingestors ('aadapp=<AZURE_APP_ID>;<AZURE_TENANT_ID>')

5. Create a table and ingestion mapping by running this: (Replace <ADE_TABLE_NAME> and <ADE_INGESTION_MAPPING>)

   .execute database script <|
   .drop table <ADE_TABLE_NAME> ifexists
   .create table <ADE_TABLE_NAME> (
      metadata: dynamic,
      transaction: dynamic
   )
   .create table <ADE_TABLE_NAME> ingestion json mapping '<ADE_INGESTION_MAPPING>' ```
   [
      { "column": "transaction", "path": "$.transaction" },
      { "column": "metadata", "path": "$.metadata" }
   ]
   ```

   Feel free to add more columns to the table and ingestion json mapping

Use the following env vars to setup:

Export transactions to json file.

Use the following env vars to setup:

Export transactions as a POST request to a web address.

The transactions will be sent as a JSON array in the body of the request with the following structure:

{
    date: string, // "dd/mm/yyyy"
    amount: number,
    description: string,
    memo: string,
    category: string,
    account: string,
    hash: string,
    comment: string | undefined,
    "scraped at": string, // "YYYY-MM-DD"
    "scraped by": string,
    identifier: string,
    chargedCurrency: string | undefined,
}

Use the following env vars to setup:

WIP

1. Follow the instructions here to create a google service account.
2. Create a new sheet and share it with your service account using the GOOGLE_SERVICE_ACCOUNT_EMAIL.

Use the following env vars to setup:

To export your transactions directly to YNAB you need to use the following environment variables to setup:

A JSON key-value pair structure representing a mapping between two identifiers. The key represent the account ID as is understood by moneyman and the value it's the UUID visible in the YNAB URL when an account is selected.

For example, in the URL: https://app.ynab.com/22aa9fcd-93a9-47e9-8ff6-33036b7c6242/accounts/ba2dd3a9-b7d4-46d6-8413-8327203e2b82 the account UUID is the second UUID.

Example:

{
  "5897": "ba2dd3a9-b7d4-46d6-8413-8327203e2b82"
}

To export your transactions directly to Buxfer you need to use the following environment variables to setup:

A JSON key-value pair structure representing a mapping between two identifiers. The key represent the account ID as is understood by moneyman (as obtained from web scrapping the financial institutions) and the value it's the UUID visible in the Buxfer URL when an account is selected.

For example, in the URL: https://www.buxfer.com/account?id=123456 the account UUID is the account id query parameter.

Example:

{
  "5897": "123456"
}