Current examples for creating keys, updating and getting freesites

[Michael Pearce]

Hey all.

I'm wanting to use Freenet to store small pieces of JSON data.  I'm wanting to do the following operations:

* Generate a new Public / Private Key Pair
* Insert / Update a JSON string of data using the above private key
* Get the JSON string using the public key

In the documentation for SSK, it shows an example of doing this via telnet (on port 2323, which i can't connect to): https://wiki.freenetproject.org/Signed_Subspace_Key.  I attempted the MAKESSK telnetting to port 9481 but it didn't work.  From what i can tell, this may no longer be the preferred way to do things.

I then looked at the FCP docs at https://wiki.freenetproject.org/FCPv2, which i'm gonna guess is what i should be doing.  I then tried the following:

~/projects/ $ telnet 127.0.0.1 9481
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GenerateSSK
Identifier=My Identifier Blah Blah
EndMessage
Connection closed by foreign host.

It does not look to have generated a key.

Is there any example that i may have missed in my googling?

Thanks!

-mike

[Michael Pearce]

Spam detection software, running on the system "freenetproject.org", has
identified this incoming email as possible spam. The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email. If you have any questions, see
the administrator of that system for details.

Content preview: Hey all. I missed the Hello message. My previous command works
after issuing: ClientHello Name=My Freenet Client ExpectedVersion=2.0 EndMessage
-mgp [...]

Content analysis details: (5.6 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
trust
[98.138.91.66 listed in list.dnswl.org]
1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
[Blocked - see <http://www.spamcop.net/bl.shtml?98.138.91.66>]
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
(michaelgpearce[at]yahoo.com)
0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is
CUSTOM_MED
0.0 HTML_MESSAGE BODY: HTML included in message
-0.5 BAYES_05 BODY: Bayes spam probability is 1 to 5%
[score: 0.0155]
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid
3.0 NOT_SIGNED_BUT_EXPECTED Message should have been signed by DKIM but
wasn't
2.0 FREENET_LOC_SHORT Contains short body and URI
1.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
-1.3 AWL AWL: From: address is in the auto white-list

The original message was not completely plain text, and may be unsafe to
open with some email clients; in particular, it may contain a virus,
or confirm that your address can receive spam. If you wish to view
it, it may be safer to save it to a file and open it with an editor.

[Steve Dougherty]

Post by Michael Pearce
Hey all.
I'm wanting to use Freenet to store small pieces of JSON data. I'm
* Generate a new Public / Private Key Pair
* Insert / Update a JSON string of data using the above private key
* Get the JSON string using the public key
In the documentation for SSK, it shows an example of doing this via
https://wiki.freenetproject.org/Signed_Subspace_Key. I attempted the
MAKESSK telnetting to port 9481 but it didn't work. From what i can
tell, this may no longer be the preferred way to do things.
I then looked at the FCP docs at
https://wiki.freenetproject.org/FCPv2, which i'm gonna guess is what
~/projects/ $ telnet 127.0.0.1 9481 Trying 127.0.0.1... Connected to
localhost. Escape character is '^]'. GenerateSSK Identifier=My
Identifier Blah Blah EndMessage Connection closed by foreign host.
It does not look to have generated a key.
Is there any example that i may have missed in my googling?
Thanks!
-mike

Sorry for the confusion! It looks like there's a bug. In this case the
node should have responded before closing the connection:

ProtocolError
Fatal=true
CodeDescription=ClientHello must be first message
Code=1
Global=false
EndMessage

So to generate an SSK keypair:

$ telnet localhost 9481
...
ClientHello
Name=TestingStuff
ExpectedVersion=2.0
End

The node will give a NodeHello.

GenerateSSK
End

The identifier field is not required for GenerateSSK. [1]

That said, there's no particular need to build your own library unless
you'd like to. For instance, there is lib-pyFreenet for Python. [2]
Inserting and fetching files are basic operations, and I hope would not
require special work to use.

-Steve

[0] https://wiki.freenetproject.org/FCPv2/ClientHello
[1] https://wiki.freenetproject.org/FCPv2/GenerateSSK
[2] https://github.com/freenet/lib-pyFreenet-staging

[Michael Pearce]

Hi Steve.

I'm using Ruby and the library for Ruby is... well... a little out of date.  There were a couple of quick changes to make (syntax for Ruby 1.9 and library changes), but i was having a hard time getting things to work with making a blocking request...  Since i really want to do just a few simple things, i'll most likely roll my own library and publish as a Ruby Gem.

I did have another question:
How do applications such as a "chat" work?  The only way that I could think of is to have multiple clients sharing the same SSK private key, publish to a USK to write a new message, then read all versions of with the public key to see what others have written.  Is this even close?

Post by Michael Pearce
Hey all.
I'm wanting to use Freenet to store small pieces of JSON data.  I'm
* Generate a new Public / Private Key Pair
* Insert / Update a JSON string of data using the above private key
* Get the JSON string using the public key
In the documentation for SSK, it shows an example of doing this via
https://wiki.freenetproject.org/Signed_Subspace_Key. I attempted the
MAKESSK telnetting to port 9481 but it didn't work.  From what i can
tell, this may no longer be the preferred way to do things.
I then looked at the FCP docs at
https://wiki.freenetproject.org/FCPv2, which i'm gonna guess is what
~/projects/ $ telnet 127.0.0.1 9481 Trying 127.0.0.1... Connected to
localhost. Escape character is '^]'. GenerateSSK Identifier=My
Identifier Blah Blah EndMessage Connection closed by foreign host.
It does not look to have generated a key.
Is there any example that i may have missed in my googling?
Thanks!
-mike

Sorry for the confusion! It looks like there's a bug. In this case the
node should have responded before closing the connection:

ProtocolError
Fatal=true
CodeDescription=ClientHello must be first message
Code=1
Global=false
EndMessage

So to generate an SSK keypair:

$ telnet localhost 9481
...
ClientHello
Name=TestingStuff
ExpectedVersion=2.0
End

The node will give a NodeHello.

GenerateSSK
End

The identifier field is not required for GenerateSSK. [1]

That said, there's no particular need to build your own library unless
you'd like to. For instance, there is lib-pyFreenet for Python. [2]
Inserting and fetching files are basic operations, and I hope would not
require special work to use.

-Steve

[0] https://wiki.freenetproject.org/FCPv2/ClientHello
[1] https://wiki.freenetproject.org/FCPv2/GenerateSSK

[2] https://github.com/freenet/lib-pyFreenet-staging

[Steve Dougherty]

Post by Michael Pearce
Hi Steve.
I'm using Ruby and the library for Ruby is... well... a little out of
date. There were a couple of quick changes to make (syntax for Ruby
1.9 and library changes), but i was having a hard time getting things
to work with making a blocking request... Since i really want to do
just a few simple things, i'll most likely roll my own library and
publish as a Ruby Gem.

Alright, sounds good.

Post by Michael Pearce
I did have another question: How do applications such as a "chat"
work? The only way that I could think of is to have multiple clients
sharing the same SSK private key, publish to a USK to write a new
message, then read all versions of with the public key to see what
others have written. Is this even close?

That could work, but it would be problematic. With this kind of approach
one malicious person can spam messages to that single key so that no one
can reliably send messages.

The problem to solve here is finding other identities to fetch messages
from. My understanding is that FMS and FLIP (examples of forums and chat
respectively) have a shared key for announcing identities, and after
that each identity inserts messages to their own private SSK.
Individuals can then be ignored at will, and all a malicious individual
can do - at least at a key level - is overwhelm the announcement
mechanism. This is often coupled with a trust/scoring system where in
addition to publishing messages, identities publish a trust list, and
give bad scores to spammers or those who abuse their trust list.

If you want to do something like this, you could roll your own identity
announcement, or make use of an established system like Web of Trust
(WoT) or Less Crappy Web of Trust (LCWoT). [0][1] WoT provides identity
creation and announcement. Unlike FMS and FLIP, it does not use a shared
announcement key. Instead it ships with a list of trusted "seed
identities" that already run the software, and add at neutral trust
those who solve CAPTCHAs they publish. LCWoT was written due to
frustration with WoT's performance and stability, and only supports
established identities.

-Steve

[0] https://github.com/freenet/plugin-WoT-staging/
[1] https://github.com/tmarkus/LessCrappyWebOfTrust

[Michael Pearce]

Thanks for the response!  All good information.  Its fun to think about how to build a system with the very specific constraints that Freenet imposes.

I'm not building a messaging application per se, but users will need to communicate and most identities will not be very long-lived.  This is leading me towards thinking I can get away with a single key to announce new messages.  I was also thinking about adding a Nonce to the message combined with an expensive hashing algorithm (i.e. BCrypt) so spamming a user takes some work on the spammer's side. 

A sender would create a message similar to:

Sender <- my public identifier

Recipient <- recipient's public identifer
Nonce <- unique value to this recipient from the sender for this message

SenderSignature <- sender signs message with (recipient, nonce) to ensure sender is who he says he is
AntiSpamHash <- Bcrypt(recipient, nonce, minimum_rounds) to force some work on the sender

MessageBodyId <- location of signed/encrypted message body located at a different Freenet key

As a receiver checking for incoming messages, he only fetches the message body for messages that:

* Have himself as the recipient
* Have a unique Nonce from the sender (no duplicates to avoid signature / anti-spam value stealing)
* Correct signature at SenderSignature
* Correct AntiSpamHash

If all of these match, the receiver will then retrieve the message body.

I think the above scheme will allow for looking at the shared key pretty quickly, and a computationally expensive AntiSpamHash will disincentivize large amounts of spam.  Definitely not perfect but it may be good enough for my needs.

Post by Michael Pearce
Hi Steve.
I'm using Ruby and the library for Ruby is... well... a little out of
date.  There were a couple of quick changes to make (syntax for Ruby
1.9 and library changes), but i was having a hard time getting things
to work with making a blocking request...  Since i really want to do
just a few simple things, i'll most likely roll my own library and
publish as a Ruby Gem.

Alright, sounds good.

Post by Michael Pearce
I did have another question: How do applications such as a "chat"
work?  The only way that I could think of is to have multiple clients
sharing the same SSK private key, publish to a USK to write a new
message, then read all versions of with the public key to see what
others have written.  Is this even close?

That could work, but it would be problematic. With this kind of approach
one malicious person can spam messages to that single key so that no one
can reliably send messages.

The problem to solve here is finding other identities to fetch messages
from. My understanding is that FMS and FLIP (examples of forums and chat
respectively) have a shared key for announcing identities, and after
that each identity inserts messages to their own private SSK.
Individuals can then be ignored at will, and all a malicious individual
can do - at least at a key level - is overwhelm the announcement
mechanism. This is often coupled with a trust/scoring system where in
addition to publishing messages, identities publish a trust list, and
give bad scores to spammers or those who abuse their trust list.

If you want to do something like this, you could roll your own identity
announcement, or make use of an established system like Web of
Trust
(WoT) or Less Crappy Web of Trust (LCWoT). [0][1] WoT provides identity
creation and announcement. Unlike FMS and FLIP, it does not use a shared
announcement key. Instead it ships with a list of trusted "seed
identities" that already run the software, and add at neutral trust
those who solve CAPTCHAs they publish. LCWoT was written due to
frustration with WoT's performance and stability, and only supports
established identities.

-Steve

[0] https://github.com/freenet/plugin-WoT-staging/

[1] https://github.com/tmarkus/LessCrappyWebOfTrust

[Michael Pearce]

I just noticed... the AntiSpamHash should also include the sender's identifier to avoid the sender using a nonce from a message already sent to the recipient and avoiding calculating the AntiSpamHash.



On Tuesday, November 12, 2013 1:07 AM, Michael Pearce <michaelgpearce-/***@public.gmane.org> wrote:

Thanks for the response!  All good information.  Its fun to think about how to build a system with the very specific constraints that Freenet imposes.

I'm not building a messaging application per se, but users will need to communicate and most identities will not be very long-lived.  This is leading me towards thinking I can get away with a single key to announce new messages.  I was also thinking about adding a Nonce to the message combined with an expensive hashing algorithm (i.e. BCrypt) so spamming a user takes some work on the spammer's side. 

A sender would create a message similar to:

Sender <- my public identifier

Recipient <- recipient's public identifer
Nonce <- unique value to this recipient from the sender for this message

SenderSignature <- sender signs message with (recipient, nonce) to ensure sender is who he says he is
AntiSpamHash <- Bcrypt(recipient, nonce, minimum_rounds) to force some work on the sender

MessageBodyId <- location of signed/encrypted message body located at a different Freenet key

As a receiver checking for incoming messages, he only fetches the message body for messages that:

* Have himself as the recipient
* Have a unique Nonce from the sender (no duplicates to avoid signature / anti-spam value stealing)
* Correct signature at SenderSignature
* Correct AntiSpamHash

If all of these match, the receiver will then retrieve the message body.

I think the above scheme will allow for looking at the shared key pretty quickly, and a computationally expensive AntiSpamHash will disincentivize large amounts of spam.  Definitely not perfect but it may be good enough for my needs.

Post by Michael Pearce
Hi Steve.
I'm using Ruby and the library for Ruby is... well... a little out of
date.  There were a couple of quick changes to make (syntax for Ruby
1.9 and library changes), but i was having a hard time getting things
to work with making a blocking

request...  Since i really want to do

Post by Michael Pearce
just a few simple things, i'll most likely roll my own library and
publish as a Ruby Gem.

Alright, sounds good.

Post by Michael Pearce
I did have another question: How do applications such as a "chat"
work?  The only way that I could think of is to have multiple clients
sharing the same SSK private key, publish to a USK to write a new
message, then read all versions of with the public key to see what
others have written.  Is this even close?

That could work, but it would be problematic. With this kind of approach
one malicious person can spam messages to that single key so that no one
can reliably send messages.

The problem to solve
here is finding other identities to fetch messages
from. My understanding is that FMS and FLIP (examples of forums and chat
respectively) have a shared key for announcing identities, and after
that each identity inserts messages to their own private SSK.
Individuals can then be ignored at will, and all a malicious individual
can do - at least at a key level - is overwhelm the announcement
mechanism. This is often coupled with a trust/scoring system where in
addition to publishing messages, identities publish a trust list, and
give bad scores to spammers or those who abuse their trust list.

If you want to do something like this, you could roll your own identity
announcement, or make use of an established system like Web of
Trust
(WoT) or Less Crappy Web of Trust (LCWoT). [0][1] WoT provides identity
creation and announcement. Unlike FMS and FLIP, it does not use a shared
announcement key. Instead it ships with a list of trusted "seed
identities" that already run the software, and add at neutral trust
those who solve CAPTCHAs they publish. LCWoT was written due to
frustration with WoT's performance and stability, and only supports
established identities.

-Steve

[0] https://github.com/freenet/plugin-WoT-staging/

[1] https://github.com/tmarkus/LessCrappyWebOfTrust

[Steve Dougherty]

Post by Michael Pearce
I just noticed... the AntiSpamHash should also include the sender's
identifier to avoid the sender using a nonce from a message already
sent to the recipient and avoiding calculating the AntiSpamHash.

That does seem like an improvement, but one worry I have with
short-lived identities in general is that a spammer needs only to
compute more hashes faster to spam more effectively. It's a balancing
act between making identity establishment fast enough to be useful and
involving enough social forces / slowness to impair spam.

Post by Michael Pearce
On Tuesday, November 12, 2013 1:07 AM, Michael Pearce
Thanks for the response! All good information. Its fun to think
about how to build a system with the very specific constraints that
Freenet imposes.
I'm not building a messaging application per se, but users will need
to communicate and most identities will not be very long-lived. This
is leading me towards thinking I can get away with a single key to
announce new messages. I was also thinking about adding a Nonce to
the message combined with an expensive hashing algorithm (i.e.
BCrypt) so spamming a user takes some work on the spammer's side.

May I ask what you're building? It will be easier to give more informed
and specific suggestions with that knowledge.

Post by Michael Pearce
Sender <- my public identifier
Recipient <- recipient's public identifer Nonce <- unique value to
this recipient from the sender for this message
SenderSignature <- sender signs message with (recipient, nonce) to
ensure sender is who he says he is AntiSpamHash <- Bcrypt(recipient,
nonce, minimum_rounds) to force some work on the sender
MessageBodyId <- location of signed/encrypted message body located at
a different Freenet key
As a receiver checking for incoming messages, he only fetches the
* Have himself as the recipient
* Have a unique Nonce from the sender (no duplicates to avoid
signature / anti-spam value stealing)
* Correct signature at SenderSignature * Correct AntiSpamHash
If all of these match, the receiver will then retrieve the message body.
I think the above scheme will allow for looking at the shared key
pretty quickly, and a computationally expensive AntiSpamHash will
disincentivize large amounts of spam. Definitely not perfect but it
may be good enough for my needs.

This is between individual users? This sounds a lot like Freemail's use
case. Perhaps you could base your protocol on that, or use Freemail as a
transport? [0]

Steve

[0] https://github.com/Thynix/Freemail/blob/spec/docs/spec/spec.tex

[Michael Pearce]

My plan is to also associate a messaging identity to a Bitcoin address with a non-zero balance that the user owns (i.e. can sign with).  This should help ensure that there is something closer to a person attached to the sender.  Nothing will be perfect in regards to spam, but i'm hopeful this will get me close enough.

Thanks a lot for the link to the Freemail spec.  I'm sure i wouldn't have come across this on my own and gives a good use case of working with Freenet as a communication layer.

Post by Michael Pearce
I just noticed... the AntiSpamHash should also include the sender's
identifier to avoid the sender using a nonce from a message already
sent to the recipient and avoiding calculating the AntiSpamHash.

That does seem like an improvement, but one worry I have with
short-lived identities in general is that a spammer needs only to
compute more hashes faster to spam more effectively. It's a balancing
act between making identity establishment fast enough to be useful and
involving enough social forces / slowness to impair spam.

Post by Michael Pearce
On Tuesday, November 12, 2013 1:07 AM, Michael Pearce
Thanks for the response!  All good information.  Its fun to think
about how to build a system with the very specific constraints that
Freenet imposes.
I'm not building a messaging application per se, but users will need
to communicate and most identities will not be very long-lived.  This
is leading me towards thinking I can get away with a single key to
announce new messages.  I was also thinking about adding a Nonce to
the message combined with an expensive hashing algorithm (i.e.
BCrypt) so spamming a user takes some work on the spammer's side.

May I ask what you're building? It will be easier to give more informed
and specific suggestions with that knowledge.

Post by Michael Pearce
Sender <- my public identifier
Recipient <- recipient's public identifer Nonce <- unique value to
this recipient from the sender for this message
SenderSignature <- sender signs message with (recipient, nonce) to
ensure sender is who he says he is AntiSpamHash <- Bcrypt(recipient,
nonce, minimum_rounds) to force some work on the sender
MessageBodyId <- location of signed/encrypted message body located at
a different Freenet key
As a receiver checking for incoming messages, he only fetches the
* Have himself as the recipient
* Have a unique Nonce from the sender (no duplicates to avoid
  signature / anti-spam value stealing)
* Correct signature at SenderSignature * Correct AntiSpamHash
If all of these match, the receiver will then retrieve the message body.
I think the above scheme will allow for looking at the shared key
pretty quickly, and a computationally expensive AntiSpamHash will
disincentivize large amounts of spam.  Definitely not perfect but it
may be good enough for my needs.

This is between individual users? This sounds a lot like Freemail's use
case. Perhaps you could base your protocol on that, or use Freemail as a
transport? [0]


Steve

[0] https://github.com/Thynix/Freemail/blob/spec/docs/spec/spec.tex

[Steve Dougherty]

Post by Michael Pearce
My plan is to also associate a messaging identity to a Bitcoin
address with a non-zero balance that the user owns (i.e. can sign
with). This should help ensure that there is something closer to a
person attached to the sender. Nothing will be perfect in regards to
spam, but i'm hopeful this will get me close enough.
Thanks a lot for the link to the Freemail spec. I'm sure i wouldn't
have come across this on my own and gives a good use case of working
with Freenet as a communication layer.

Cool! That sounds interesting indeed. At first glance it sounds like
something that could be achieved by signing the WoT ID with the Bitcoin
address and adding that as a WoT context/property. Do you plan to do
other things that make this not applicable?

[Michael Pearce]

I've actually decided to do a pivot and do a little bit less ambitious first project - but it will still include Bitcoin and still include FreeNet.  I want to get to know some of the pitfalls and problem areas before embarking on the larger project since I'll be using a few technologies I haven't used before.  I'll let this list know when i've got more questions and hopefully something for people to try out.

Post by Michael Pearce
My plan is to also associate a messaging identity to a Bitcoin
address with a non-zero balance that the user owns (i.e. can sign
with).  This should help ensure that there is something closer to a
person attached to the sender.  Nothing will be perfect in regards to
spam, but i'm hopeful this will get me close enough.
Thanks a lot for the link to the Freemail spec.  I'm sure i wouldn't
have come across this on my own and gives a good use case of working
with Freenet as a communication layer.

Cool! That sounds interesting indeed. At first glance it sounds like
something that could be achieved by signing the WoT ID with the Bitcoin
address and adding that as a WoT context/property. Do you plan to do
other things that make this not applicable?