It was wrongly refering to equiprobable distribution, which I think
I'd originally done, but forgot to update the comment after changing
to triangular
Reported by smooth on IRC
Two recent papers quantified the real usage bias for the
real output in a ring being the true one, and shows that
the current biasing is much too weak.
While we wait for a better solution, we increase the ratio
of recent-to-total fake outputs, as well as decrease the
time window for recent outputs, so that half the fake outs
are selected within the last 1.8 day. Value plucked from
figure 10, page 11 of An Empirical Analysis of Linkability
in the Monero Blockchain, 2017, Miller et al.
This is also arbitrary, of course, but serves as a stopgap
till a better selection algorithm is chosen.
If using a large input and many destinations, the code would
generate as many outputs as it could using that input, even if
it would bring the resulting tx above the max tx size.
With the change from the original transfer method to the new
algorithm, payments to the same destination were merged. It
seemed like a good idea, optimizing space. However, it is a
useful tool for people who want to split large outputs into
several smaller ones (ie, service providers making frequent
payments, and who do not like a large chunk of their balance
being locked for 10 blocks after each payment).
Default to off, which is a change from the previous behavior.
When a single input is enough to satisfy a transfer, the code would
previously try to add a second input, to match the "canonical" makeup
of a transaction with two inputs and two outputs. This would cause
wallets to slowly merge outputs till all the monero ends up in a
single output, which causes trouble when making two transactions
one after the other, since change is locked for 10 blocks, and an
increasing portion of the remaining balance would end up locked on
each transaction.
There are two new settings (min-output-count and min-output-value)
which can control when to stop adding such unneeded second outputs.
The idea is that small "dust" outputs will still get added, but
larger ones will not.
Enable with, eg:
set min-output-count 10
set min-output-value 30
to avoid using an unneeded second output of 30 monero or more, if
there would be less than 10 such outputs left.
This does not invalidate any other reason why such outputs would
be used (ie, when they're really needed to satisfy a transfer, or
when randomly picked in the normal course of selection). This may
be improved in the future.
Asking for a full histogram from a remote node (since it's
untrusted) is pretty slow, and spams the remote node, so
we replace it by only adding a second input if we have rct
ones, which are for all intents and purposes always mixable.
Minimum mixin 4 and enforced ringct is moved from v5 to v6.
v5 is now used for an increased minimum block size (from 60000
to 300000) to cater for larger typical/minimum transaction size.
The fee algorithm is also changed to decrease the base per kB
fee, and add a cheap tier for those transactions which we do
not care if they get delayed (or even included in a block).
This reverts commit d47dac9a88.
Callers actually expect the key to be payment id, so this
needs a lot more changes (like storing payment ids in the
structure, and possibly also to other existing structures
which do the same thing).
7a44f38a Add support for the wallet to refresh pruned blocks (moneromooo-monero)
da18898f ringct: do not require range proof in decodeRct/decodeRctSimple (moneromooo-monero)
b49c6ab4 rpc: add a default category for daemon rpc (moneromooo-monero)
f113b92b core: add functions to serialize base tx info (moneromooo-monero)
6fd4b827 node_rpc_proxy: allow caching daemon RPC version (moneromooo-monero)
b5c74e40 wallet: invalidate node proxy cache when reconnecting (moneromooo-monero)
A relatedness check was meant to be done in the case of adding
an extra output if just one was enough. This was mistakenly
added to the "preferred output" case.
c02e1cb9 Updates to epee HTTP client code - http_simple_client now uses std::chrono for timeouts - http_simple_client accepts timeouts per connect / invoke call - shortened names of epee http invoke functions - invoke command functions only take relative path, connection is not automatically performed (Lee Clagett)
0644eed7 Remove boost/foreach.cpp includes (Miguel Herranz)
36dd3e23 Replace BOOST_REVERSE_FOREACH with ranged for (Miguel Herranz)
629e3101 Replace BOOST_FOREACH with C++11 ranged for (Miguel Herranz)
If a rct transaction can be made with just one input, a second
output will be added. This output will be the smallest amount
output available. However, if this output is a non rct output
with less available fake outs than requested, the transaction
will be rejected. We now check the histogram to only consider
outputs with enough available fake outs in the first place.
These warnings were emitted by clang++, and they are real bugs.
src/rpc/core_rpc_server.cpp:208:58: warning: adding 'uint64_t'
(aka 'unsigned long') to a string does not append to the string
[-Wstring-plus-int]
res.status = "Error retrieving block at height " + height;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
The obvious intent is achieved by using std::to_string().
- http_simple_client now uses std::chrono for timeouts
- http_simple_client accepts timeouts per connect / invoke call
- shortened names of epee http invoke functions
- invoke command functions only take relative path, connection
is not automatically performed
This would have tried to send a second output to make the tx
look like the 2/2 ideal, but it would not fail to find one
because picking an output from preferred_inputs priority list
did not remove it from the unused tranfer/dust outputs, so
it would try to send the same output twice.
While there, I also added a check to avoid sending a second
input if it's related to the first. Better 1/2 than linking
inputs, I think.
This avoids indirectly leaking the real output to the daemon,
and is faster.
This will still happen for more complex cases, especially
when cancelling a tx and "re-rolling" it.
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.
To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:
This one is (mostly) silent, only outputting fatal errors:
MONERO_LOGS=*:FATAL
This one is very verbose:
MONERO_LOGS=*:TRACE
This one is totally silent (logwise):
MONERO_LOGS=""
This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):
MONERO_LOGS=*:WARNING,verify:FATAL
Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE
Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:
MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE
Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.
Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.
The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
If we'd make a rct tx with just one input, we try to add
a second one to match the 2/2 ideal. This means more txes
use that template (and are thus using a larger anonymity
set), and it coalesces outputs "for free". We use the
smallest amount outputs in priority for this, so we can
"clean" the wallet at the same time.
c2135082 simplewallet: add a show_transfer <txid> command (moneromooo-monero)
19c4041d wallet_rpc_server: new RPC call to get a transfer by txid (moneromooo-monero)
07b9138c support importing unportable outputs (kenshi84)
2ac80075 also use portable serializer for boost_serialization_helper.h and net_node.inl, completely adandon boost/archive/binary_oarchive.hpp (kenshi84)
d1d6e27a moved boost cpp into hpp since they're supposed to be header only (kenshi84)
66e6af89 added experimental boost::archive::portable_binary_{i|o}archive (kenshi84)
944b6079 Wallet API: Do not refresh while daemon is syncing + fixed fast refresh when creating wallet offline + improved close wallet logic (make sure refresh thread is stopped) (Jaquee)
A bug in cold signing caused a spurious pubkey to be included
in transactions, so we need to ensure we use the correct one
when sending outputs from one of those.
If a rct transaction would cause no change to be generated, a zero
change output is added, and sent to a randomly generated address.
This ensures that no transaction will be sent with just one output,
which could cause the receiver to be able to determine which of the
inputs in the sent rings is the real one.
This is very rare, since it requires the sum of outputs to be equal
to the sum of outputs plus the fee, which is now a function of the
last few blocks.
5783dd8c tests: add unit tests for uri parsing (moneromooo-monero)
82ba2108 wallet: add API and RPC to create/parse monero: URIs (moneromooo-monero)
d9001b43 epee: add functions to convert from URL format (ie, %XX values) (moneromooo-monero)
Daemon RPC version is now composed of a major and minor number,
so that incompatible changes bump the major version, while
compatible changes can still bump the minor version without
causing clients to unnecessarily complain.
6d76072 simplewallet: remove double confirmation when submitting signed tx (moneromooo-monero)
92dea04 wallet2: fix wrong change being recorded for cold signed txes (moneromooo-monero)
1d9e223 rpc: do not include output indices for pool txes (moneromooo-monero)
e227d6e rpc: bump version after RPC changes (moneromooo-monero)
2c0173c Add a get_outs (fully text based) version of get_outs.bin (moneromooo-monero)
e05907b rpc: add output indices to gettransactions (moneromooo-monero)
When passing around unsigned and signed transactions, outputs
and key images are passed along (outputs are passed along unsigned
transactions from the hot wallet to the cold wallet, key images
are passed along with signed transations from the cold wallet
to the hot wallet), to allow more user friendly syncing between
hot and cold wallets.
The vast majority of transactions will have just one tx pubkey,
but a bug with cold wallet signing caused two such keys to be
there, with the second one being the real one.
View wallets do not have the spend secret key, and are thus
unable to derive key images for incoming outputs. Moreover,
a previous patch set key images to zero as a means to mark
an output as having an unknown key image, so they could be
filled in when importing key images at a later time. That
later patch caused spurious collisions. We now use public
keys to detect duplicate outputs. Public keys obtained from
the blockchain are checked to be identical to the ones
derived locally, so can't be spoofed.
m_amount_out was sometimes getting initialized with the sum of
an transaction's outputs, and sometimes with the sum of outputs
that were not change. This caused confusion and bugs. We now
always set it to the sum of outputs. This reverts an earlier
fix for bad amounts as this used the other semantics. The wallet
data should be converted automatically in a percentage of cases
that I'm hesitant to estimate. In any case, restoring from seed
or keys or rebuilding the cache will get it right.
Compute derivation only once per tx, instead of once per output. Approx 33% faster while using 75% as much CPU on my machine. Note old functions in cryptonote_core (lookup_acc_outs and is_out_to_acc) are still used by tests.
The intended use is to export outputs from a hot wallet, which
can scan incoming transfers from the network, and import them
in the cold wallet, which can't. The cold wallet can then compute
key images for those outputs, which can then be exported with
export_key_images, etc.
Re-creating the transaction on the cold wallet was not splitting
the change, causing the transaction to be rejected by the network.
This worked on testnet since amounts do not have to be split.
Also add selected_transfers, which can now be saved since they're
size_t rather than iterators. This allows the view wallet to
properly set the sent outputs as spent and update balance.
Bump transfer file version numbers to match.
0950be9 wallet: speed up output selection, and fix bug with relatedness calculation (moneromooo-monero)
0eba133 wallet: fix mixup between mixin 2 and 4 before/after v5 (moneromooo-monero)
25% of the outputs are selected from the last 5 days (if possible),
in order to avoid the common case of sending recently received
outputs again. 25% and 5 days are subject to review later, since
it's just a wallet level change.
This was still using the old transaction creation algorithm,
coupled with a deterministic output selection scheme, which
made it ill suited to the job, since it'd loop indefinitely
in case the fee increased between the test tx and adding the
fee.
8b20cbf libwallet_api: do not use fast-refresh on recovery (Ilya Kitaev)
10fe626 libwallet_api: fast-refresh in case of opening non-synced wallet (Ilya Kitaev)
0019e31 libwallet_api: fix unhandled exception on address check (Ilya Kitaev)
1f73f80 libwallet_api: fast-refresh for new wallet (Ilya Kitaev)
4789347 libwallet_api: test for create/init wallet on mainnet (Ilya Kitaev)
bba6af9 wallet: cold wallet transaction signing (moneromooo-monero)
9872dcb wallet: fix log confusion between bytes and kilobytes (moneromooo-monero)
d9b0bf9 cryptonote_core: make extra field removal more generic (moneromooo-monero)
98f19d4 serialization: add support for serializing std::pair and std::list (moneromooo-monero)
This change adds the ability to create a new unsigned transaction
from a watch only wallet, and save it to a file. This file can
then be moved to another computer/VM where a cold wallet may load
it, sign it, and save it. That cold wallet does not need to have
a blockchain nor daemon. The signed transaction file can then be
moved back to the watch only wallet, which can load it and send
it to the daemon.
Two new simplewallet commands to use it:
sign_transfer (on the cold wallet)
submit_transfer (on the watch only wallet)
The transfer command used on a watch only wallet now writes an
unsigned transaction set in a file called 'unsigned_monero_tx'
instead of submitting the tx to the daemon as a normal wallet does.
The signed tx file is called 'signed_monero_tx'.
This is intended to catch traffic coming from a web browser,
so we avoid issues with a web page sending a transfer RPC to
the wallet. Requiring a particular user agent can act as a
simple password scheme, while we wait for 0MQ and proper
authentication to be merged.
This fixes misreporting of amount/fee in rct txes, as the rct
tx construction code was lumping all dests (whether change or
not) in the same dests vector, while the pre-rct code was
keeping it separate.
Keep the immediate direct deps at the library that depends on them,
declare deps as PUBLIC so that targets that link against that library
get the library's deps as transitive deps.
Break dep cycle between blockchain_db <-> crytonote_core.
No code refactoring, just hide cycle from cmake so that
it doesn't complain (cycles are allowed only between
static libs, not shared libs).
This is in preparation for supproting BUILD_SHARED_LIBS cmake
built-in option for building internal libs as shared.
We keep 1, 2, 3 multipliers till the fee decrase from 0.01/kB
to 0.002/kB, where we start using 1, 20, 166 multipliers.
This ensures the higher multiplier will compensate for the
block reward penalty when pushing past 100% of the past median.
The fee-multiplier wallet setting is now rename to priority,
since it keeps its [0..3] range, but maps to different multiplier
values.
CMake issued a warming about policy CMP0026: access of LOCATION
target property at config time was disallowed. Offending code
was the code that merged static libraries to generate
libwallet_merged.a.
This patch does that same merge task in a much simpler way. And,
since it doesn't violate the policy, the warning went away.
When RingCT is enabled, outputs from coinbase transactions
are created as a single output, and stored as RingCT output,
with a fake mask. Their amount is not hidden on the blockchain
itself, but they are then able to be used as fake inputs in
a RingCT ring. Since the output amounts are hidden, their
"dustiness" is not an obstacle anymore to mixing, and this
makes the coinbase transactions a lot smaller, as well as
helping the TXO set to grow more slowly.
Also add a new "Null" type of rct signature, which decreases
the size required when no signatures are to be stored, as
in a coinbase tx.
This allows the key to be not the same for two outputs sent to
the same address (eg, if you pay yourself, and also get change
back). Also remove the key amounts lists and return parameters
since we don't actually generate random ones, so we don't need
to save them as we can recalculate them when needed if we have
the correct keys.
This plugs a privacy leak, where the wallet tells the daemon
which transactions contain outputs for the wallet by asking
for additional information for that particular transaction.
As a nice bonus, this actually makes refresh slightly faster.
With RCT, we allow 0 size outputs, to try and encourage txes
with two inputs and two outputs. Consolidation would then
have two non zero inputs, one zero output, and one larger
output.
Before the normal selection, we attempt to find either one or two
suitable outputs to use as inputs to the rct tx. The intent is that
most rct txes will have one or two inputs, and we want all to look
the same if possible.
When two outputs are needed, we try to find a pair which are not
related (ie, by being from the same or similar block height).
The "transfer" simplewallet command is renamed to "transfer_original".
"transfer_new" is renamed "transfer", "transfer_rct" is removed,
and the new "transfer" now selects rct or non rct transactions
based on the current block height.
The mixRing (output keys and commitments) and II fields (key images)
can be reconstructed from vin data.
This saves some modest amount of space in the tx.
If the blockchain gets reorganized, all outputs spent in the part
of the blockchain that's blown away need to be reset to unspent
(they may end up spent again on the blocks that replace the blocks
that are removed, however).
99dd572 libwallet_api: tests: checking for result while opening wallet (Ilya Kitaev)
bcf7b67 libwallet_api: Wallet::amountFromString fixed (Ilya Kitaev)
32bc7b4 libwallet_api: helper method to return maximumAllowedAmount (Ilya Kitaev)
cbe534d libwallet_api: tests: removed logged passwords (Ilya Kitaev)
b1a5a93 libwallet_api: do not store wallet on close if status is not ok (Ilya Kitaev)
This plugs a privacy leak from the wallet to the daemon,
as the daemon could previously see what input is included
as a transaction input, which the daemon hadn't previously
supplied. Now, the wallet requests a particular set of
outputs, including the real one.
This can result in transactions that can't be accepted if
the wallet happens to select too many outputs with non standard
unlock times. The daemon could know this and select another
output, but the wallet is blind to it. It's currently very
unlikely since I don't think anything uses non default
unlock times. The wallet requests more outputs than necessary
so it can use spares if any of the returns outputs are still
locked. If there are not enough spares to reach the desired
mixin, the transaction will fail.
This constrains the number of instances of any amount
to the unlocked ones (as defined by the default unlock time
setting: outputs with non default unlock time are not
considered, so may be counted as unlocked even if they are
not actually unlocked).
They are used to export a signed set of key images from a wallet
with a private spend key, so an auditor with the matching view key
may see which of those are spent, and which are not.
It is not clear why libunbound was added to this in the first place,
since it wasn't here before and #915 doesn't seem to introduce any
new dependency on it.
Tested build with STATIC=OFF (with and without libunbound-dev libunbound8
installed) and STATIC=ON, on Ubuntu Trusty, Debian Jessie, and Arch
Linux. For static builds, beware of #926 and #907.
If this hack was introduced to make it build on some other system
(Windows? OS X?), then it will have to be dealt with, but not this way.
Signing is done using the spend key, since the view key may
be shared. This could be extended later, to let the user choose
which key (even a per tx key).
simplewallet's sign/verify API uses a file. The RPC uses a
string (simplewallet can't easily do strings since commands
receive a tokenized set of arguments).
