Commit Graph

1796 Commits

Author SHA1 Message Date
Howard Chu
b2f1c58805 Use cursors in some remove functions
Helps when they're called repeatedly in one txn
2016-04-05 20:57:31 +01:00
Howard Chu
118dd69dd5 Use DUPFIXED for block_info and output_txs
Saves another ~150MB or so on the full blockchain
2016-04-05 20:55:16 +01:00
Howard Chu
6225716f3c More outputs consolidation
Also bumped DB VERSION to 1
Another significant speedup and space savings:
Get rid of global_output_indices, remove indirection from output to keys

This is the change warptangent described on irc but never got to finish.
2016-04-05 20:55:12 +01:00
Howard Chu
7c5abdc3a3 Use DUPFIXED for output_keys
Saves another 90MB on 200000 block import.
Had to bring back compare_uint64 for this, but it's safe since
this table is always 64-bit aligned.
2016-04-05 20:54:42 +01:00
Howard Chu
8e9d8e3364 Use DUPFIXED for tx_indices
Small space savings, no measurable speedup
2016-04-05 20:54:39 +01:00
Howard Chu
a12f9365f8 Use DUPFIXED for block_heights
Only a small savings...
2016-04-05 20:54:07 +01:00
Howard Chu
38c2277d6f Use DUPFIXED for spent_keys 2016-04-05 20:54:07 +01:00
warptangent
bf769c32ba Add back changes from revert.
m_tx_outputs doesn't need to be changed, as it's no longer dup list.
2016-04-05 20:54:07 +01:00
warptangent
46b991b362 Use MDB_APPEND mode with two tx subdbs
This is possible on those using a tx index as a key.
2016-04-05 20:54:07 +01:00
warptangent
9aadedb1d0 Schema update: tx_indices - consolidate the tx subdbs from 5 to 3 2016-04-05 20:54:06 +01:00
warptangent
a2f518aa01 Schema update: tx_indices - yet less indirection 2016-04-05 20:54:06 +01:00
warptangent
8d12a8df2c Schema update: tx_indices - improve further with less indirection 2016-04-05 20:54:06 +01:00
warptangent
7c013f66e9 Add batch warning for further review 2016-04-05 20:54:06 +01:00
warptangent
ae0854a431 Schema update: tx_indices 2016-04-05 20:54:06 +01:00
Howard Chu
8d252a4214 Consolidated block info 2016-04-05 20:53:59 +01:00
warptangent
132c666f67 Update schema for "tx_outputs" to use array containing amount output indices
This speeds up wallet refresh by directly retrieving a tx's amount output indices.

It removes the indirection and walking the amount output duplicate list
for every amount in each requested tx.

"tx_outputs" is used by:
Amount output indices are needed for wallet refresh.
Global output indices are needed for removing a tx.

Both amount output indices and global output indices are now stored in
an array of 64-bit unsigned ints:

tx_outputs[<tx_hash>] -> [ <a1_oi, a1_gi, a2_oi, a2_gi, ...> ]

Previously it was:
tx_outputs[<tx_hash>] -> duplicate list of <a1_gi, a2_gi, a3_gi, ...>

The amount output list had to be walked for every amount in order to
find each amount's output index, by comparing the amount's global output
index with each one in the duplicate list until a match was found.

See also d045dfa7ce
2016-04-05 20:30:50 +01:00
warptangent
309f8f3d44 blockchain_utilities: Update documentation 2016-04-05 20:30:50 +01:00
luigi1111
7ac1db42c2 get_payments short ID
Add support for short/integrated/encrypted IDs to get_payments RPC
2016-04-05 20:30:50 +01:00
Riccardo Spagni
38fed83b65
Merge pull request
dfca2ae unit_tests: fix hard fork unit test compilation (warptangent)
2016-03-05 23:12:32 +02:00
Riccardo Spagni
93605eb05f
Merge pull request
0485d17 blockchain_export: Support BerkeleyDB (warptangent)
41487e3 blockchain_export: Move DB implementation includes (warptangent)
2016-03-05 23:12:05 +02:00
Riccardo Spagni
32dc08d19a
Merge pull request
bdec7cb BlockchainLMDB: Use DB error helper consistently (warptangent)
c5932eb BlockchainLMDB: Add DB error to exception (warptangent)
a49c355 Blockchain: Omit verbose time stats messages by default (warptangent)
2016-03-05 23:10:38 +02:00
Riccardo Spagni
c3af15702f
Merge pull request
ee7a8b8 Get rid of lmdb_cur (Howard Chu)
2016-03-05 23:06:26 +02:00
Riccardo Spagni
14c1e214ad
Merge pull request
d57ec75 Fix language (Howard Chu)
2016-03-05 23:06:04 +02:00
Riccardo Spagni
80aa57a6cd
Merge pull request
f5affbe More for df239428c0 (Howard Chu)
2016-03-05 23:05:29 +02:00
warptangent
dfca2ae1b5
unit_tests: fix hard fork unit test compilation
Match changed BlockchainDB function declaration.
2016-03-05 10:01:10 -08:00
warptangent
0485d17274
blockchain_export: Support BerkeleyDB
TEST:

blockchain_export -h

This should show "berkeley" as an available option to --database.

With an existing BerkeleyDB database, run:

blockchain_export --database berkeley
2016-03-05 06:46:11 -08:00
warptangent
41487e32be
blockchain_export: Move DB implementation includes 2016-03-05 06:46:09 -08:00
warptangent
bdec7cbfb3
BlockchainLMDB: Use DB error helper consistently 2016-03-03 21:27:13 -08:00
warptangent
c5932eb5ea BlockchainLMDB: Add DB error to exception 2016-03-03 20:15:37 -08:00
warptangent
a49c355cae
Blockchain: Omit verbose time stats messages by default
This is already the default for the daemon, but by checking a command
line argument and calling a Blockchain member function setter.

Initialize the variable to false so it's not dependent on an external
command-line argument check. This allows utilities like
blockchain_import to have a reasonable default without code changes.
2016-03-03 20:14:48 -08:00
Howard Chu
ee7a8b87f9 Get rid of lmdb_cur
We don't need it now with per-txn cursors.
2016-03-03 19:06:27 +00:00
Howard Chu
d57ec7595c Fix language 2016-03-03 00:34:19 +00:00
Howard Chu
f5affbef26 More for df239428c0
Let ARMv7 work again
2016-03-03 00:13:13 +00:00
Riccardo Spagni
a7310031f4
Merge pull request
2c823c1 Make partial revert consistent (warptangent)
2016-03-01 07:40:00 -05:00
warptangent
2c823c1589
Make partial revert consistent
If user-defined comparator is used, subdb shouldn't be opened with
MDB_INTEGERKEY.

TODO: Again, this will be added back with future schema updates.
2016-03-01 02:47:16 -08:00
Riccardo Spagni
e0493e8429
Merge pull request
df23942 Revert part of 7db89ed2ee (warptangent)
c9c4060 BlockchainLMDB: Add height and db error to exception (warptangent)
2016-02-29 21:09:51 -05:00
warptangent
df239428c0
Revert part of 7db89ed2ee
For now, so existing databases work.

TODO: add these back with future schema updates.
2016-02-29 18:03:41 -08:00
warptangent
c9c4060dc6
BlockchainLMDB: Add height and db error to exception 2016-02-29 17:52:54 -08:00
Riccardo Spagni
95b3b09e11
Merge pull request
5ab33ca unit_tests: update test openalias address (moneromooo-monero)
2016-02-29 17:31:07 -05:00
Riccardo Spagni
bf09285443
Merge pull request
c2a1fee simplewallet: prompt for private keys when generating wallets (moneromooo-monero)
4513b4c simplewallet: add a new --restore-from-keys option (moneromooo-monero)
2016-02-29 17:30:29 -05:00
Riccardo Spagni
c3c4e28061
Merge pull request
5e3557d move g_test_dbg_lock_sleep from a global to a function level static (moneromooo-monero)
2016-02-29 17:30:10 -05:00
Riccardo Spagni
d60bf4ee36
Merge pull request
7db89ed ARMv7: fix unaligned accesses (Howard Chu)
5a07cef Wrap some more actions in a larger read txn (Howard Chu)
8cc7a36 read txn/cursor stuff (Howard Chu)
86a7f2b core: check whether an update is needed straight away (moneromooo-monero)
ea5fa5e core: print "update needed" hard fork notifications in red (moneromooo-monero)
2016-02-29 17:29:26 -05:00
Riccardo Spagni
838f7aba7d
Merge pull request
283894d dns_utils: const compatibility with older libunbound (moneromooo-monero)
2016-02-29 17:28:48 -05:00
moneromooo-monero
c2a1fee428
simplewallet: prompt for private keys when generating wallets
instead of supplying them on the command line. It's more
unwieldy, but better protects against information leaks.
2016-02-27 11:53:11 +00:00
Howard Chu
7db89ed2ee ARMv7: fix unaligned accesses
And cleanup some key comparators
2016-02-25 13:57:00 +00:00
moneromooo-monero
5ab33cabf6
unit_tests: update test openalias address
It was recently changed
2016-02-23 23:12:55 +00:00
Howard Chu
5a07cefe7b Wrap some more actions in a larger read txn 2016-02-23 20:47:15 +00:00
Howard Chu
8cc7a36f0b read txn/cursor stuff
Could wrap more later.
2016-02-23 20:47:15 +00:00
moneromooo-monero
86a7f2b1e7 core: check whether an update is needed straight away 2016-02-23 20:29:17 +00:00
moneromooo-monero
ea5fa5e9b6 core: print "update needed" hard fork notifications in red 2016-02-23 20:29:17 +00:00