Discussion:
Solution to churn problem and possibly to the Pitch Black attack
Pierre Abbat
2011-05-15 05:14:31 UTC
Permalink
I read the Pitch Black paper and I think I've figured out a solution. Instead
of trying to fit a circular keyspace in a graph that's spread over the
surface of the earth with some long-distance links, I think it would be
better to make the keyspace have more dimensions. Here's my proposal:

Break a key or location into four equal parts. These are coordinates of the
point on a four-dimensional torus. Distance is measured in 8-space by
embedding each torus dimension as a circle.

Every 1024 times a node adds something to its store, it performs this
calculation: First it adds all the keys in its store as vectors in 8-space,
then reduces that to a 4-dimensional point on the torus. Then it adds all its
neighbors' locations and again reduces it to a point on the torus. The result
is its new location. The location will be close to its neighbors; if the
surface containing locations is curved, more of the store will be on the
outside of the curve. Young nodes with few keys stored will be pulled rapidly
toward their neighbors; old nodes will remain stably located near their data.
Unlike the current method, where the product of distances rewards nodes for
having locations next to each other, in this system nodes next to each other
will be pulled apart, spreading more evenly over the keyspace.

The number of dimensions could be any divisor of 32 (the number of bytes in a
hash). More than that and the notion of averaging in a dimension becomes
problematic. I have a hunch that the optimal number of dimensions is 4, where
two represent the surface of the earth and the other two different social
classes in a region. Could someone who has a Freenet simulator run some tests
and see how well this method performs for various numbers of dimensions?

Pierre
--
I believe in Yellow when I'm in Sweden and in Black when I'm in Wales.
Matthew Toseland
2011-05-24 12:56:07 UTC
Permalink
Post by Pierre Abbat
I read the Pitch Black paper and I think I've figured out a solution. Instead
of trying to fit a circular keyspace in a graph that's spread over the
surface of the earth with some long-distance links, I think it would be
Break a key or location into four equal parts. These are coordinates of the
point on a four-dimensional torus. Distance is measured in 8-space by
embedding each torus dimension as a circle.
Every 1024 times a node adds something to its store, it performs this
calculation: First it adds all the keys in its store as vectors in 8-space,
then reduces that to a 4-dimensional point on the torus. Then it adds all its
neighbors' locations and again reduces it to a point on the torus. The result
is its new location. The location will be close to its neighbors; if the
surface containing locations is curved, more of the store will be on the
outside of the curve. Young nodes with few keys stored will be pulled rapidly
toward their neighbors; old nodes will remain stably located near their data.
Unlike the current method, where the product of distances rewards nodes for
having locations next to each other, in this system nodes next to each other
will be pulled apart, spreading more evenly over the keyspace.
The number of dimensions could be any divisor of 32 (the number of bytes in a
hash). More than that and the notion of averaging in a dimension becomes
problematic. I have a hunch that the optimal number of dimensions is 4, where
two represent the surface of the earth and the other two different social
classes in a region. Could someone who has a Freenet simulator run some tests
and see how well this method performs for various numbers of dimensions?
You should simulate this. There are a few simulators but you might have to write your own.

The main difficulty I see is that greedy routing only works if most of our connections are close and a few are distant. If they are all distant, a request will get to roughly where it should be quickly but will not be able to get to a specific location in a reasonable number of hops.
Pierre Abbat
2011-05-24 14:18:50 UTC
Permalink
Post by Matthew Toseland
You should simulate this. There are a few simulators but you might have to write your own.
The main difficulty I see is that greedy routing only works if most of our
connections are close and a few are distant. If they are all distant, a
request will get to roughly where it should be quickly but will not be able
to get to a specific location in a reasonable number of hops.
To run a simulator, do I need to know Java? I have two computers, one a
years-old Ubuntu installation that I haven't had time to upgrade, and one a
recent DragonFly box that I've had trouble installing Java on. The Ubuntu
box's Java interpreter reports the following:

java version "1.5.0"
gij (GNU libgcj) version 4.2.1 (Ubuntu 4.2.1-5ubuntu5)

Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

It's good enough to run camxes, but I don't know about running Freenet.

Also I have lots of work to do including three jobs and designing my house. I
may not be able to get around to running the simulator until September at
least.

Pierre
--
Don't buy a French car in Holland. It may be a citroen.
Ian Clarke
2011-05-25 17:38:20 UTC
Permalink
Post by Pierre Abbat
Post by Matthew Toseland
You should simulate this. There are a few simulators but you might have
to
Post by Matthew Toseland
write your own.
The main difficulty I see is that greedy routing only works if most of
our
Post by Matthew Toseland
connections are close and a few are distant. If they are all distant, a
request will get to roughly where it should be quickly but will not be
able
Post by Matthew Toseland
to get to a specific location in a reasonable number of hops.
To run a simulator, do I need to know Java? I have two computers, one a
years-old Ubuntu installation that I haven't had time to upgrade, and one a
recent DragonFly box that I've had trouble installing Java on. The Ubuntu
My recommendation would be to whip up a simulation in whatever language you
are most familiar with. It shouldn't be complicated, and will probably be
easier than the learning curve of using unfamiliar code in an unfamiliar
language.

This is very interesting work!

Ian.
--
Ian Clarke
Founder, The Freenet Project
Email: ian-***@public.gmane.org
Pierre Abbat
2011-05-26 14:55:51 UTC
Permalink
Post by Ian Clarke
My recommendation would be to whip up a simulation in whatever language you
are most familiar with. It shouldn't be complicated, and will probably be
easier than the learning curve of using unfamiliar code in an unfamiliar
language.
Writing a simulator from scratch is too big a job for me. I found one by Hui
Zhang in C++ which I'll see if I can modify. If you know of others, please
let me know.

Pierre
--
Jews use a lunisolar calendar; Muslims use a solely lunar calendar.
Matthew Toseland
2011-05-31 23:16:01 UTC
Permalink
Post by Pierre Abbat
Post by Ian Clarke
My recommendation would be to whip up a simulation in whatever language you
are most familiar with. It shouldn't be complicated, and will probably be
easier than the learning curve of using unfamiliar code in an unfamiliar
language.
Writing a simulator from scratch is too big a job for me. I found one by Hui
Zhang in C++ which I'll see if I can modify. If you know of others, please
let me know.
There are several, but make sure you are running it at the right level. It sounds like you're only interested in routing and swapping (or similar mechanisms). You don't need accurate simulations of requests and packets.
Post by Pierre Abbat
Pierre
Pierre Abbat
2011-06-01 04:33:39 UTC
Permalink
Post by Matthew Toseland
There are several, but make sure you are running it at the right level. It
sounds like you're only interested in routing and swapping (or similar
mechanisms). You don't need accurate simulations of requests and packets.
There's no mention of a level in the source code. Where are the other
simulators? There's also no swapping that I could see in the simulator.

Pierre
--
The Black Garden on the Mountain is not on the Black Mountain.
Matthew Toseland
2011-06-01 14:09:58 UTC
Permalink
Post by Pierre Abbat
Post by Matthew Toseland
There are several, but make sure you are running it at the right level. It
sounds like you're only interested in routing and swapping (or similar
mechanisms). You don't need accurate simulations of requests and packets.
There's no mention of a level in the source code. Where are the other
simulators? There's also no swapping that I could see in the simulator.
Good question. I'm not entirely sure, some of them are in our git repositories.
https://github.com/freenet/

One extreme option is to use many nodes in a single JVM. This gives accurate but very slow simulations. See src/freenet/node/simulator/ in freenet's source for this.

You will probably have to write your own though. :|
Post by Pierre Abbat
Pierre
Loading...