Planet Primates

July 22, 2014

StackOverflow

Scala graph and problems when declaring custom types

right now I have a package object which has several types declared to make my code a bit easier to read.

    package object FlowGraphType {
type FlowEdgeType[NodeLabel] = LDiEdge[NodeLabel]
type FlowGraph = Graph[NodeLabel, FlowEdgeType]

type DominatorGraph = Graph[NodeLabel, FlowEdgeType]
type DominatorEdgeType[NodeLabel] = DiEdge[NodeLabel]

type NodeLabel = Long
val FlowTaken = 'taken
val FlowFallthrough = 'fallthrough

}


This works fine:

    val graph = Graph[NodeLabel, FlowEdgeType]((1L ~+> 2L)(FlowTaken), (2L ~+> 4L)(FlowTaken))


However, when I try to use the generic type to declare the graph, like below, I get the message not found: value FlowGraph

    val graph = new FlowGraph((1L ~+> 2L)(FlowTaken), (2L ~+> 4L)(FlowTaken))


When I try new like this, I get trait Graph is abstract; cannot be instantiated

    val graph = new FlowGraph((1L ~+> 2L)(FlowTaken), (2L ~+> 4L)(FlowTaken), (2L ~+> 3L)(FlowTaken), (3L ~+> 5L)(FlowTaken), (4L ~+> 5L)(FlowTaken), (5L ~+> 6L)(FlowTaken))


I'm a bit new to Scala, so decoding what exactly it going wrong here is a bit difficult for me. My first guess was that the apply method of Graph which is a factory isn't being called when the type is declared in place of the custom type.

Would love any insights into this. Thanks a bunch!

Edit So this allows me to actually use my type with the constructor:

      type FlowGraph = Graph[NodeLabel, FlowEdgeType]
val FlowGraph = Graph[NodeLabel, FlowEdgeType]_


However, now I must type 'Seq' in order to get the list properly processed. Is there a way to omit the Seq and pass those values to the original varargs?

      val graph1 = FlowGraph(Seq((1L ~+> 2L)(FlowTaken)))


Edit 2 type FlowGraph = Graph[NodeLabel, FlowEdgeType] val FlowGraph = Graph[NodeLabel, FlowEdgeType]_ def FlowGraph(xs: FlowEdgeType[NodeLabel]) : FlowGraph = FlowGraph(Seq(xs: _))

Is there any comprehensive tutorial on how to use this kind of structural typing?

As seen here, map is defined

in a slightly funky way

For completeness' sake, here's the code:

implicit def FunctionFunctor[R] = new Functor[({type l[a] = R=>a})#l] {
def map[A, B](fa: R => A)(f: A => B) : R => B = (x => f(fa(x)))
}


More concretely - new Functor[({type l[a] = R=>a})#l]

I do think I know what's going on, but can't say honestly that I fully understand the concept. And since there is no any hint, I can't event google the term (I simply don't know that to google). Does some tutorial (or review, or whatever) exist where this is explained with a better level of detalization? I'd appreciate even more is someone could explain it right here in the answers.

CompsciOverflow

Combinatory interpretation of lambda calculus

According to Peter Selinger, The Lambda Calculus is Algebraic (PDF). Early in this article he says:

The combinatory interpretation of the lambda calculus is known to be imperfect, because it does not satisfy the ξ-rule: under the interpretation, M = N does not imply λx.M = λx.N (Barendregt, 1984).

Questions:

• What kind of equivalence is meant here?
• Given this definition of equivalence, what is a counter-example of the implication?

StackOverflow

Clojure Compojure Ring and https

I'm using Clojure (ring and compojure) to build a web app and I don't know where to start with https. I have a registration and login that will need to be secured, then once they're authenticated they'll need to stay in https.

I can't seem to find a good tutorial on setting up https in general or for a clojure app specifically.

I found this answer: How do you enable https and http->https redirects in ring / compojure

Does that mean I can write my compojure app as if there's no https and have nginx sit in front and take care of all that for me?

Can't resolve casbah as a sbt dependency

I'm getting the following error upon compile:

[error] (*:update) sbt.ResolveException: unresolved dependency: org.mongodb#casbah_2.11;2.7.4-SNAPSHOT: not found


With this build.sbt file:

name := """play-scala"""

version := "1.0-SNAPSHOT"

lazy val root = (project in file(".")).enablePlugins(PlayScala)

scalaVersion := "2.11.1"

libraryDependencies ++= Seq(
jdbc,
anorm,
cache,
ws
)

libraryDependencies += "org.mongodb" %% "casbah" % "2.7.4-SNAPSHOT"

// For stable releases
resolvers += "Sonatype releases" at "https://oss.sonatype.org/content/repositories/releases"

// For SNAPSHOT releases
resolvers += "Sonatype snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"


I'm using Typesafe Activator and have tried many combinations of Casbah versions (including releases, instead of snapshots) and resolvers. The installation instructions from the "official" MongoDB Casbah tutorial don't work either. Any ideas?

How can I convert a java.io.Reader to a string in Scala? [duplicate]

This question already has an answer here:

If I have a java.io.Reader instance, how can I convert it to a string in Scala?

val reader = new java.io.Reader()
...
// How do I get a String out of reader?


Return all the indexes of a particular substring

Is there a Scala library API method (and if not, an idiomatic way) to obtain a list of all the indexes for a substring (target) within a larger string (source)? I have tried to look through the ScalaDoc, but was not able to find anything obvious. There are SO many methods doing so many useful things, I am guessing I am just not submitting the right search terms.

For example, if I have a source string of "name:Yo,name:Jim,name:name,name:bozo" and I use a target string of "name:", I would like to get back a List[Int] of List(0, 8, 17, 27).

Here's my quick hack to resolve the problem:

def indexesOf(source: String, target: String, index: Int = 0, withinOverlaps: Boolean = false): List[Int] = {
def recursive(index: Int, accumulator: List[Int]): List[Int] = {
if (!(index < source.size)) accumulator
else {
val position = source.indexOf(target, index)
if (position == -1) accumulator
else {
recursive(position + (if (withinOverlaps) 1 else target.size), position :: accumulator)
}
}
}

if (target.size <= source.size) {
if (!source.equals(target)) {
recursive(0, Nil).reverse
}
else List(0)
}
else Nil
}


Any guidance you can give me replacing this with a proper standard library entry point would be greatly appreciated.

UPDATE 2014/Jul/22:

Inspired by Siddhartha Dutta's answer, I tighted up my code. It now looks like this:

  def indexesOf(source: String, target: String, index: Int = 0, withinOverlaps: Boolean = false): List[Int] = {
@tailrec def recursive(indexTarget: Int, accumulator: List[Int]): List[Int] = {
val position = source.indexOf(target, indexTarget)
if (position == -1) accumulator
else {
recursive(position + (if (withinOverlaps) 1 else target.size), position :: accumulator)
}
}
recursive(index, Nil).reverse
}


Additionally, if I have a source string of "aaaaaaaa" and I use a target string of "aa", I would like by default to get back a List[Int] of List(0, 2, 4, 6) which skips a search starting inside of a found substring. The default can be overridden by passing "true" for the withinOverlaps parameter which in the "aaaaaaaa"/"aa" case would return List(0, 1, 2, 3, 4, 5, 6).

CompsciOverflow

Recovering a point embedding from a graph with edges weighted by point distance

Suppose I give you an undirected graph with weighted edges, and tell you that each node corresponds to a point in 3d space. Whenever there's an edge between two nodes, the weight of the edge is the distance between the points.

Your goal is to reconstruct the relative positions of the points, given only the available distances (represented by the edge weights). For example, if I gave you $d_{0,1} = d_{0,2} = d_{0,3} = d_{1,2} = d_{1,3} = d_{2,3} = 1$, then you know the points are the vertices of a tetrahedron. You don't know where it is relative to the origin, or its orientation, or if it's been mirrored, but you can tell it's a tetrahedron.

In general, the problem is easy if I give you all of the edge lengths. Just arbitrarily pick a point $p_0$ to be at $(0,0,0)$, then pick a neighboring point $p_1$ and place it at $(d_{0,1},0,0)$, then a common neighbor $p_2$ gets triangulated onto the XY plane, then a final common neighbor $p_3$ gets triangulated into the half-space $z > 0$ and breaks the remaining symmetry (assuming you didn't pick degenerate points). You can use those four points to triangulate all the remaining ones.

On the other hand, when some edge lengths are missing it may not be possible to recover the embedding. For example, if there's a vertex that disconnects the graph when cut, then the two components it would separate if removed can swing around relative to each other.

Which raises the questions:

• How expensive is it to find a solution?
• How do you determine if a solution is unique, up to translation/rotation/mirroring? Is 3-connectedness sufficient? Necessary?
• What sorts of conditions make the problem trivial?
• If I don't promise the edge weights actually correspond to point distance sin 3d, how expensive is it to determine if an embedding is possible at all?

Longest Path A*, Admissible Heuristics, and Optimalness

Im working on a modified A Star algorithm, that instead of attempting to find the shortest path, it tries to find the longest past, or in my specific case, the highest weighted path.

I have some number of classes, each with a list of unique nodes, with each node having its own value. Im attempting to determine the best possible combination of a node from each class to determine the combination or path that has the greatest weight. To do this, I am viewing these classes as a tree, with each Level having the nodes contained within a single class.

Then I search through this tree using A Star, or more specifically, im searching through my tree based on a search stack. Once a node is explored, its children are inserted in a sorted order based on their weight (plus their ancestors weight) plus the possible future weight (my heuristic). Then the top of the stack, with the highest value is selected to search next.

To do this I have an overestimating heuristic, that is it never underestimates the the optimal solution.

If I am looking for the highest weight and not the lowest weight, is this heuristic admissible, and thus my algorithm optimal?

PS: A FormalIsh defination of the current algorithm.

Let S = {S1, S2, ... , Sn}

and each Si has a set of items and a NULL, which represents an item not chosen from this set.

Si = {I1, I2, ... , Im, NULL}

Also each item only ever exists in one set, IE Si U Sj = Si + Sj

Each Item, Ii has an associated value, Vi.

The problem is to select a maximum of M offers, one from each set, when summed yield the highest value. I call this selection a Path, Pi. A path can be complete, meaning it has a selection from all S, or it can be partial where only x offers are contained within it. Also M

In addition, there exists a function IsCompatable(path) that returns true if a path is compatible. Whether a path is compatible or is completely arbitrary. The Max valued path must be compatible. This is why I can not trivially select the M largest Items from each set.

In addition each set contains a NULL item, so that an item need not be selected from that set.

The Trivial Algorithm would to make a search tree and generate all possible combinations of items in S, with each path to the trees leaves said to be a path.

let G(P) be the current value (the summed value of each item) of the partial path. Let H(P) be an estimation (heuristic) of the value of the future path. H(P) = The Y values from Y items from Y Si in S. Each item is the item with the maximum value in the Si where i > len(P). Y = M - the current length of the partial path P.

To Find the path with the greatest value, I keep a sorted Queue of partial paths, sorted on their values + their possible future values, ie G(Pi) + H(Pi). I Initialize this Queue by adding paths contained in S1.

While the Queue is not empty or a path has not been found:
p = Pop the path from the top of Q
if p is Complete:
A full path has been found
return p
find the possible children of p by adding an item to p from the next set.
for child in Children:
if IsCompatable(child):
add child back to Q in sorted order on G(child) + H(child)


There it is, now is my Heuristic admissible?

StackOverflow

How to use custom plugin in multi-module project?

In a multi-module project, with one module implementing a custom SBT plugin with custom TaskKey, how this plugin can be imported for the project settings of another submodule.

If the submodule using the plugin is define in submodule1/build.sbt, then submodule1/project/plugins.sbt is not loaded.

If plugin is registered in project/plugins.sbt it will fails when loading top/aggregate project as plugin is not necessarily already built.

Is there any other way to define a custom task requiring custom dependency so that it can be used by a submodule?

TheoryOverflow

Resource listing models with known VC dimension

Is there any reference resource gathering models with known VC dimension? I am looking for an exhaustive list of models with their VC dimension (and ideally the associated proof or a pointer to it).

It would be in the same spirit as the Complexity Zoo, which gathers most of the known complexity classes, or the textbook Computers and Intractability, which catalogues many NP-Complete problems.

CompsciOverflow

Looking for an algorithm to solve a specific Vehicle Routing Problem

Hopefully I have come to the right place...

I am trying to figure out a way to create routes for trucks to complete a list of orders(drops/stops), while minimizing distance traveled.

• There is only ever 1 company warehouse in the area.
• The trucks have to deliver based on capacity.
• Each truck can hold a maximum (usually 18 pallets).
• Each order will be for a number underneath that maximum.
• There will be a maximum number of trucks specified.
• When trucks are finished with their route, they will return to the company warehouse.

I already have all of the orders, the pallets they are requesting, and the distance between each point.

I am an absolute simpleton when it comes to complex problems like these... I am hoping that someone has a simple (relatively) solution, or an article of some sort that could help me down my path.

Thank you all very much for your time!

StackOverflow

call java class in spark pi?

I have been using spark using scala fairly successfully. The driver code uses other scala classes inside RDD.

Now I want to switch to python. Can I still use these external scala class or do I need to rewrite them in python classes?

Functional Programming Performance

I've recently started to use Scala to solve some programming challenges at Codeforces in order to exercise functional programming skills. Doing so I came across one particular challenge I wasn't able to solve in a way respecting the given execution time limit of 1000ms; the Painting Fence problem.

I tried various different ways, starting with a straight forward recursive solution, trying a similar approach using streams instead of lists, and eventually trying to reduce list manipulations by working a bit more with indices. I ended up having stack overflow exceptions on larger tests that I was able to fix using Scala's TailCall. But still, while the solution correctly solves the problem, it is too slow to complete within 1000ms. In addition to that, there's a C++ implementation shown that is ridiculously fast in comparison (< 50ms). Now I do understand that Scala will be slower compared to C++ in many cases, and I also understand that I could write a more imperative-style solution in Scala that would probably perform a lot better. Still, I'm wondering if I am missing something more fundamental here, because I have a hard time believing that functional programming is that much slower in general (and I am pretty new to functional programming).

Here is my scala code that you can paste in the REPL including the example that takes >1000ms:

import scala.util.control.TailCalls._

def solve(l: List[(Int, Int)]): Int = {

def go(from: Int, to: Int, prevHeight: Int): TailRec[Int] = {
val max = to - from
val currHeight = l.slice(from, to).minBy(_._1)._1
val hStrokes = currHeight - prevHeight
val splits = l.slice(from, to).filter(_._1 - currHeight == 0).map(_._2)
val indices = from :: splits.flatMap(x => List(x, x+1)) ::: List(to)
val subLists = indices.grouped(2).filter(xs => xs.last - xs.head > 0)

val trampolines = subLists.map(xs => tailcall(go(xs.head, xs.last, currHeight)))
val sumTrampolines = trampolines.foldLeft(done(hStrokes))((b, a) => b.flatMap(bVal =>
a.map(aVal => aVal + bVal)))
sumTrampolines.flatMap(v => done(max).map(m => Math.min(m, v)))
}
go(0, l.size, 0).result
}

val lst = (1 to 5000).toList.zipWithIndex
val res = solve(lst)


And for comparison, here's a C++ example achieving the same thing written by Bugman (includes some read/write from console that I didn't include in the Scala version above):

#include <iostream>
#include <cstdio>
#include <cstdlib>
#include <algorithm>
#include <vector>
#include <string>
#include <set>
#include <map>
#include <cmath>
#include <memory.h>
using namespace std;
typedef long long ll;

const int N = 1e6+6;
const int T = 1e6+6;

int a[N];
int t[T], d;

int rmq(int i, int j){
int r = i;
for(i+=d,j+=d; i<=j; ++i>>=1,--j>>=1){
if(i&1) r=a[r]>a[t[i]]?t[i]:r;
if(~j&1) r=a[r]>a[t[j]]?t[j]:r;
}
return r;
}

int calc(int l, int r, int h){
if(l>r) return 0;

int m = rmq(l,r);
int mn = a[m];
int res = min(r-l+1, calc(l,m-1,mn)+calc(m+1,r,mn)+mn-h);
return res;
}

int main(){
//freopen("input.txt","r",stdin);// freopen("output.txt","w",stdout);

int n, m;

scanf("%d",&n);
for(int i=0;i<n;++i) scanf("%d",&a[i]);

a[n] = 2e9;
for(d=1;d<n;d<<=1);
for(int i=0;i<n;++i) t[i+d]=i;
for(int i=n+d;i<d+d;++i) t[i]=n;
for(int i=d-1;i;--i) t[i]=a[t[i*2]]<a[t[i*2+1]]?t[i*2]:t[i*2+1];

printf("%d\n",calc(0,n-1,0));

return 0;
}


At least before I introduced the explicit tail calls, the more functional style seemed more natural to me to solve the problem than a more imperative solution. So I'd be really happy to learn more on what I should be attentive of when writing functional code in order to still get acceptable performance.

Function composition dynamically

Let' s say I have a function move.

Now, depending on some inputs (let' s say the input says "Move by 3"), I have to move 3 times.

I would do this like :

move compose move compose move


Is it possible to do function composition dynamically? So that I can build my new function depending on the number of times I have to move.

Thank you

CompsciOverflow

Computing binomial coefficients and factorials modulo a composite number

Does anyone know of a fast algorithm to compute factorials and/or binomial coefficients in general or modulo a composite number in particular (for composite moduli I am interested in the case where the factorization is not necessarily known) I know that for primes and factorials we can apply Wilson's Theorem but I was wondering if there is also a fast method for composite numbers. For binomial coefficients I was thinking of using recursion and the addition rule: $$\binom{n}{x} + \binom{n}{x+1} = \binom{n+1}{x+1}$$ somehow but I'm not sure really how to go about it. Any suggessions? Thanks.

QuantOverflow

Excellent information source on advanced machine learning / data mining based trading?

I did check the related posts, like this one here.

However, given if one already has knowledge in finance, machine learning and statistics, and wants to know something more advanced on machine learning based trading rather than the introductory. What would you recommend?

I am looking for anything related: excellent books, research papers, posts, authors, as long as it is related to machine learning and trading in general, but the thing is, really looking for the sources with quality content rather than the ads sites.

StackOverflow

Moving an element to the front of a list in Scala

I was wondering if there is a way to find an element in a list and move it to the front of the list in Scala? Is there any easy way to do this other than iterating over the list, then removing that element and then pre-pending it to the front of the list?

Tail recursive factorial in Fortran95

I decided to try implementing the factorial function in Fortran95 (f2py limitation) but my efforts are only yielding two return-type-mismatch errors.

Inspiration for solution

In Haskell, we can do something like

fac_helper n acc =
if n <= 1
then acc
else fac_helper (n - 1) (acc * n)

factorial n = fac_helper n 1


Attempted solution: fac.f95

recursive function facHelper(n, acc) result(returner)
integer::n
integer::acc
integer::returner
if (n <= 1) then
returner = acc
else
returner = facHelper(n - 1, n * acc)
endif
end function facHelper

function factorial(n)
integer::n
integer::factorial
factorial = facHelper(n, 1)
end function factorial


When GNU Fortran (GCC) 4.8.3 is used on fac.f95 as

gfortran -std=f95 ./fac.f95 -o fac

the result is:

Error: Return type mismatch of function 'fachelper' at (1) (REAL(4)/INTEGER(4))
Error: Return type mismatch of function 'factorial' at (1) (REAL(4)/INTEGER(4))


These errors appear (to the un-familiar Fortraner) as out of touch with the code of which compilation was attempted on. I am certain that there are no real numbers declared or used in the attempted solution. ...?

How would the tail-recursive factorial implementation look in Fortran95?

QuantOverflow

How can I go about applying machine learning algorithms to stock markets?

I am not very sure, if this question fits in here.

I have recently begun, reading and learning about machine learning. Can someone throw some light onto how to go about it or rather can anyone share their experience and few basic pointers about how to go about it or atleast start applying it to see some results from data sets? How ambitious does this sound?

Also, do mention about standard algorithms that should be tried or looked at while doing this.

StackOverflow

Evaluating clojure function to a quoted form

I am using a function that takes arguments as follows: (test-function '((gate 1) (gate 3) (gate 2)))

The list argument can contain any number of elements where each element is of the form (gate x) where x can be an integer from 0 to 8. I have a function, generate-gate-list, that generates lists of random length (up to 10) and content, although they are ALWAYS of the form above.

Example output of generate-gate-list: ((gate 2)), (()), ((gate 1) (gate 6)), etc.

I would like to be able to nest generate-gate-list inside test-function so that I can test a bunch of randomly generated lists without generating them all beforehand. In other words, I'd like something like this: (test-function '(generate-gate-list)) except where generate-gate-list has been evaluated. I've tried some sort of macro syntax-quote and unquoting, but that leads to resolved variables, like (user/gate 3) which screws up test-function. Here's my generate-gate-list code:

    (defn generate-gate-list []
(map symbol (repeatedly (rand-int 10) #(random-gate))))


random-gate outputs a gate element as a string, i.e "(gate 3)" or "(gate 2)".

So in short, I'd like (test-function (something-here (generate-gate-list))) or (test-function (modified-generate-gate-list)) to be equivalent to (test-function '((gate 1) (gate 4) (gate 2))) or some other arbitrary output of generate-gate-list. Thanks!

TheoryOverflow

Finding the most part of common information

Let we have strings $x$ and $y$. I want to find the most part of extracting common information of $x$ and $y$, that is string $z$ with $C(z) + C(x|z) = C(x)$, $C(z) + C(y|z) = C(y)$, $C(z) \to max$ ($C$ is Kolmogorov complexity).

Is this problem (find so $z$) computable? May be $CT(z|x,y) = 0$ ($CT$ is total complexity)

StackOverflow

Pip packages not found during Vagrant provision with Ansible

When using vagrant and Ansible to install pip requirements from a .txt file I'm experiencing the following error when running 'vagrant up'

...
Running setup.py egg_info for package south

Could not find any downloads that satisfy the requirement djangorestframework==2.3.13 (from -r /opt/app/requirements.txt (line 12))
No distributions at all found for djangorestframework==2.3.13 (from -r /opt/app/requirements.txt (line 12))

FATAL: all hosts have already failed -- aborting


This seems to happen to a random package in the requirements, and when I ssh into the Vagrant box and run 'sudo pip install -r requirements.txt' myself, the packages download and install ok.

Has anyone else experienced this issue before or knows what could be causing it?

Here's my VagrantFile http://pastebin.com/ktgStS7P

Here's my Ansible playbook.yml

---

- name: application playbook
user: vagrant
hosts: all
gather_facts: false
- name: Install app pip packages
sudo: true
pip: name={{item}} state=present
with_items:
- jinja2
- markupsafe

- name: install app specific pip packages
sudo: true
pip: requirements=/opt/app/requirements.txt


Vagrant 1.4.3
Ansible 1.5.4
VirtualBox 4.3.10


CompsciOverflow

communicational complexity calculation [on hold]

i need help regarding communication complexity of security purpose algorithm. from my basic knowledge, i think algorithm with lower computational complexity is acceptable like O(n) > O(n^2); but what about communication complexity? can anyone please help regarding this comparison?

**** If i need to choose an algorithm based on only communication complexity(cc); what should i do? for example if algo A has cc of O(n) and algo B has cc of O(n^2)..which one is better?

StackOverflow

Getting rsync in Ansible to work with Vagrant

I want to test an Ansible script using Vagrant. Everything works fine until it tries to do a rsync to the remote host:

- name: Install custom dev user settings
local_action: command rsync -ave ssh roles/common/files/home/{{ item.name }}
{{ ansible_ssh_user }}@{{ inventory_hostname }}:/#home/
with_items: dev_users
when: "{{ item.custom }} == True"
tags:
- dev_custom
- dev_users
- users


However it fails at this point - it seems to be trying to login via password but I don't know why as it should connect to Vagrant via SSH, right (I have elided some information below with ... because it mentioned keys) ?

127.0.0.1> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-1393178896.64-215542007508316 && chmod a+rx$HOME/.ansible/tmp/ansible-1393178896.64-215542007508316 && echo $HOME/.ansible/tmp/ansible-1393178896.64-215542007508316'] <127.0.0.1> REMOTE_MODULE command rsync -ave ssh roles/common/files/home/someUser vagrant@10.0.0.10:/#home/ <127.0.0.1> PUT /tmp/tmpm3BnEW TO /home/mark/.ansible/tmp/ansible-1393178896.64-215542007508316/command <127.0.0.1> EXEC /bin/sh -c 'sudo -k && sudo -H -S -p "[sudo via ansible, key=...] password: " -u root /bin/sh -c '"'"'echo SUDO-SUCCESS-...; /usr/bin/python /home/mark/.ansible/tmp/ansible-1393178896.64-215542007508316/command; rm -rf /home/mark/.ansible/tmp/ansible-1393178896.64-215542007508316/ >/dev/null 2>&1'"'"'' failed: [10.0.0.10] => (item={ ... }) => {"failed": true, "item": { ... }, "parsed": false} invalid output was: [sudo via ansible, key=...] password: [sudo via ansible, key=...] password: Sorry, try again. [sudo via ansible, key=...] password: [sudo via ansible, key=...] password: Sorry, try again. [sudo via ansible, key=...] password: [sudo via ansible, key=...] password: Sorry, try again. sudo: 3 incorrect password attempts  Any idea how to get around this? I do deploy a custom key to the box via ansible before I do this e.g.: - name: Place ansible authorized key authorized_key: user=root key="{{ lookup('file', 'root/.ssh/ansible_key.pub') }}" tags: - root  QuantOverflow reference question about portfolio optimization I know the "classical" modern portfolio theory. However I have quite a lot of different sources. It seems that there is not a book which cover this topic in a rigorous way: 1. theory 2. application 3. examples in c++ / R / matlab / ... Are there any books, which cover these points? Recently, we discussed in class a dynamic portfolio optimization problem. Now I would like to know if there is any good literature about this topic? With dynamic portfolio optimization I mean changing constraints over time. Many thanks for sharing your experiences. StackOverflow Set SSH Host IP Address on Rackspace for Ansible The Question When using the rax module to spin up servers and get inventory, how do I tell Ansible to connect to the IP address on an isolated network rather than the server's public IP? Note: Ansible is being run from a server on the same isolated network. The Problem I spin up a server in the Rackspace Cloud using Ansible with the rax module, and I add it to an isolated/private network. I then add it to inventory and begin configuring it. The first thing I do is lock down SSH, in part by telling it to bind only to the IP address given to the host on the isolated network. The catch is, that means ansible can't connect over the public IP address, so I also set ansible_ssh_host to the private IP. (This happens when I add the host to inventory.) - name: Add servers to group local_action: module: add_host hostname: "{{ item.name }}" ansible_ssh_host: "{{ item.rax.addresses.my_network_name[0].addr }}" groups: launched with_items: rax_response.success when: rax_response.action = 'create'  This works just fine on that first run of creating and configuring new instances. Unfortunately, the next time I try to connect to these servers, the connection is refused because Ansible is trying at an IP address on which SSH isn't listening. This happens because: 1. Ansible tries to connect to ansible_ssh_host... 2. But the rax.py inventory script has set ansible_ssh_host to the accessIPv4 returned by Rackspace... 3. And Rackspace has set accessIPv4 to the public IP address of the server. Now, I'm not sure what to do about this. Rackspace does allow an API call to update a server and set its accessIPv4, so I thought I could run another local_action after creating the server to do that. Unfortunately, the rax module doesn't appear to allow updating a server, and even if it did it depends on pyrax which in turn depends on novaclient, and novaclient only allows updating the name of the server, not accessIPv4. Surely someone has done this before. What is the right way to tell Ansible to connect on the isolated network when getting dynamic inventory via the rax module? CompsciOverflow What is a semantic cognitive map Based on: J. P. Carvalho, "On the Semantics and the Use of Fuzzy Cognitive Maps in Social Sciences" (WCCI, 2010 -- PDF) and Richard Dagan's web page Cognitive Mapping. A cognitive map consists of nodes representing concepts and ties indicating relationship between the concepts. In Carvalho section IV, the semantics is used in perspective of the presented model. Dagan's definition of semantic map is that it is a tree like structure branching from a main concept. Also, the answer Are semantic networks ontologies? further adds to my confusion as to what is the significant different between cognitive map where nodes are concepts and semantic map where also the nodes are concepts. I am unable to find significant difference between concept map and semantic map. Can somebody please explain in simple language? Thank you StackOverflow How to configure a fine tuned thread pool for futures? How large is Scala's thread pool for futures? My Scala application makes many millions of future {}s and I wonder if there is anything I can do to optimize them by configuring a thread pool. Thank you. Elegant way in Scala to render List of objects(columns) with List attribute(cell values) in a grid? What is the "best" way to render a List of model objects(matrix) in a Scala template such as: public class Column extends Model { public String columnLabel; @OneToMany public List cells; } public class Cell extends Model { public String rowLabel; //index public BigDecimal value; }  For the sake of this question, cells.size() and rowLabel are consistent for all column objects. The controller returns a List[Column] to the view. I have tried to convert the List to an Array with a helper: @matrix(list: List[Column]):Array = @{ var rowCount = list.head.values.size() var colCount = list.size() var m = Array.ofDim[String](rowCount,colCount) for (i <- 0 to rowCount-1) { for ( j <- 0 to colCount-1) { matrix(i)(j) = list(j).cells(i).value.toString(); } } return m; }  and then in the view: <div> @for(i <- 1 to currentPage.getList.head.values.size()) { <div class="row"> @for(j <- 1 to currentPage.getList.size()) { <div class="col-md-1">@matrix(currentPage.getList)(i)(j)</div> } </div> } </div>  but of course this is only extracting the matrix values and not the column or row labels. Is there some Scala array goodness that can be used on the List of Lists? Efficiency is important as the array size will be approx. 20 columns x 2000 rows. Or is it a better approach to have the controller return the matrix rows explicitly rather than try to convert them in the view? object X is not a member of package Y I am trying to implement an abstract Java class in Scala and in the eclipse IDE it is not showing errors but, when I try to compile it is throwing an error The abstract Java class is defined as package de.hybris.platform.webservices; import de.hybris.platform.core.model.ItemModel; public abstract class AbstractCommand<RESOURCE> extends AbstractYResponseBuilder<RESOURCE, Object, Object> implements Command<RESOURCE, Object, Object> { ... }  Code for Scala class is package com.test.scala.poc.ws import de.hybris.platform.webservices.AbstractCommand import de.hybris.platform.core.model.product.ProductModel class ProductCommand extends AbstractCommand[ProductModel] { override def execute(resourceModel: ProductModel, requestDTO: Object): ProductModel = null }  And finally, the damn error message is object webservices is not a member of package de.hybris.platform  Help resolving this is much appreciated! /r/netsec $13 to launch DDoS. $25 botnets. 3 tips for battling today's cheap crimeware. StackOverflow Habit of making private methods static to increase functional programming I got in touch with the functional programming paradigm (haskell, scala) and like the concept. I'm trying to incorporate these functional principles in my every day work. Here an example public class Functional { private final Object o1; private final Object o2; public Functional(Object o1, Object o2) { this.o1 = o1; this.o2 = o2; } /* * method has side effects */ private void method() { // o1.someChange(); // ... // o2.someChange(); } /* * method has no side effects - it only uses its parameters */ private static void method(Object o1, Object o2) { // o1.someChange(); // ... // o2.someChange(); } public void work() { method(o1, o2); } public static void main(String[] args) { Functional f = new Functional(new Object(), new Object()); f.work(); } }  I find the static method easier to maintain, also for people who did not write the code, since they just have to look at the method parameters - which can be an advantage in big classes. Another minor advantage is performance, because after compilation static methods get called with invokestatic which is slightly faster. The public methods are still kept non static, since I don't want to discard OOP/encapsulation. I'm only talking about private static methods. QUESTION So what do you think of this apprach? Esp. what are the negativ sides my new habit of making all private methods static - within reason, as long as I don't need more than 3, 4 parameters? CompsciOverflow Detecting coplanarity by given pairwise distances Consider an undirected weighted graph$G = (V,E)$, where$V \subset \mathbb{R}^3$so the points are 3D, and the weight of an edge equals the (Euclidean) distance between its endpoints. Note that we're not given the coordinates of the points in V. We may not even be given all pairwise distances, so the graph need not be complete, and it may even be sparse. Suppose we're given$k$and told that there are$k$planes such that all the vertices belong to at least one of those plane. We want to find$k$such planes, with an added restriction: In order to determine whether 4 points are coplanar given only their pairwise distances, the most straightforward method is to use the Cayley-Menger determinant. For our problem, this would require the graph to be fairly dense, since we'd need to know most of the pairwise distances to apply Cayley-Menger. The restriction is to find$k$planes without using the Cayley-Menger determinant. If this is impossible, can we obtain a proof that states this is impossible? In other words, can we prove that for any such graph$G$and given$k$, if we have enough information to find$k$planes for$G$by some means, then we have enough information to use Cayley-Menger to find$k$planes? StackOverflow ScalaTest in Intellij does not print out console messages I am running Spark tests that use ScalaTest. They are very chatty on the command line using the following command (as an aside the -Dtest= is apparently ignored - all core tests are being run..):  mvn -Pyarn -Phive test -pl core -Dtest=org.apache.spark.MapOutputTrackerSuite  There are thousands of lines of output, here is a taste: 7:03:30.251 INFO org.apache.spark.scheduler.TaskSetManager: Finished TID 4417 in 23 ms on localhost (progress: 4/4) 17:03:30.252 INFO org.apache.spark.scheduler.TaskSchedulerImpl: Removed TaskSet 38.0, whose tasks have all completed, from pool 17:03:30.252 INFO org.apache.spark.scheduler.DAGScheduler: Completed ResultTask(38, 3) 17:03:30.252 INFO org.apache.spark.scheduler.DAGScheduler: Stage 38 (apply at Transformer.scala:22) finished in 0.050 s 17:03:30.288 INFO org.apache.spark.ui.SparkUI: Stopped Spark web UI at http://localhost:4041 17:03:30.289 INFO org.apache.spark.scheduler.DAGScheduler: Stopping DAGScheduler  However in IJ only tests Pass/Fail are printed out. So how to view the same chatty INFO level output as on command line? Apache Spark: distinct doesnt work? Here is my code example:  case class Person(name:String,tel:String){ def equals(that:Person):Boolean = that.name == this.name && this.tel == that.tel} val persons = Array(Person("peter","139"),Person("peter","139"),Person("john","111")) sc.parallelize(persons).distinct.collect  It returns  res34: Array[Person] = Array(Person(john,111), Person(peter,139), Person(peter,139))  Why distinct doesn't work?I want the result as Person("john",111),Person("peter",139) QuantOverflow How to deal with extreme cases in normal random numbers generation? In order to generate normal random numbers, one usually generates random numbers following a uniform distribution$Z \sim \mathcal{U}(0,1)$and then applies the reverse CDF function on them$X=\Phi^{-1}(Z) \sim \mathcal{N}(0,1)$. However, I encountered a problematic case when one of the generated$Z$turns out to give exactly 0. Then, you have$X=\Phi^{-1}(Z)=- \infty$. This is pretty problematic when you generated random samples because it will usually break all your variance/covariance measure basically returning nan or inf when the samples contain infinite number. How do you usually handle this? Do you check after each generated random number whether the value is 0 or 1 and shift it slightly (or simply dicard it)? /r/compsci Term Paper on Computer Graphics Hi reddit, I am planning out my paper on computer graphics. It'll be 15 pages when it's all said and done. I am writing to explore the effects of different color choices have on media. Namely with a focus on Video Games. I thought it would be a good idea to lead the paper through a narrative of a graphic artist/designer who is making the color choices for an upcoming release. anyways I was wondering if you, reddit, could give me some inspiration to write on. I have most of the paper planned out, but if you could throw some buzz words at me (or hopefully 1 sentence) to inspire me for a few details to cover, that would be just great. Ex: "You could talk about x." heres what I have so far: • Video games • VR • colors that are more pleasant to the users • more or less eye stress (saturation/brightness) • best choices for a HUD to not class with the world • change the color to darker to contrast a lightened backdrop • 3D environments • main characters and backgrounds • colors use for leading the story along • color choices for hinting on where/what to do (leading the user) • Graphic Design • best color choices for different categories(menu choices) • best use of colors for level of importance • box art, don't color, spacing + sizing • CGI/Films • this chapter may be incorporated into the 3D environments chapter • Sound and color • this encompasses all other topics • describe moods (color and sound relationships) • Synesthesia Thanks for your time! submitted by nsuetv14 [link] [4 comments] StackOverflow Why does Scala sbt update every time I start it from command prompt? I am a beginner in Scala and every time I start sbt from command prompt it goes into a long update and it takes several minutes for the sbt to update. I read in the instructions that the sbt was going to update the first time it was started from the command prompt but in my case it is updating every time I start it. I have to wait several minutes for it to update and it takes away some of the joy of learning Scala. So my question is how do I stop sbt from updating every time and is it normal for it to do so? edit: This is how my sbt command looks and after this it goes into a long update. edit2: Will upgrading fix the FreeBSD 8.1 portstree? I have a machine running FreeBSD and the ports tree is accidentaly updated to one of FreeBSD 8 stable. The problem is that my FreeBSD 8.1 is not configured to use PKG for packages so I cannot update or install packages. I want to upgrade the machines FreeBSD but get the ports tree in working order before I do that. It seems the only solution is installing an old 8.1 version of the ports tree, but I cannot find any. Can someone tell me where to find this? Would upgrading to the lastest stable version 8 also solve my problem? metrics clojure ring middleware not recording metrics In my handler I have the following defined: (def reg (new-registry)) (def app (-> app-routes (expose-metrics-as-json) (instrument reg)))  If I go to /metrics I only see an empty json object. What am I missing? Also, the documetation is not clear as to how I can report on the console, it seems report-to-console which is in the documentation is not used any more, instead there are reporters but I cannot find any documentation as to how you use reporters. /r/emacs Quickly create Emacs themes. StackOverflow LZW TIFF decoding When decoding tiff files with LZW decompression, the first 9 bits in the encoded bitstream should be "256", the clear-code. But when I read it I get a 128, which I just can't figure out. I created the file with GDAL. My code reading the file is: val res = (for { i <- 0 until next if (bitSet.get(i + index)) } yield (1 << i)).sum  The index is the index in the encoded bitstream and next is how many bits I should read (starting with 9). So my question is why do I read an 128 instead of an 256? When printing the bitstream input the first bit that is set as 1 is bit number 8 (index 7). The file in question is: https://dl.dropboxusercontent.com/u/42266515/aspect_lzw.tif Thanks! Spark Streaming Window Operation The following is simple code to get the word count over a window size of 30 seconds and slide size of 10 seconds. import org.apache.spark.SparkConf import org.apache.spark.streaming._ import org.apache.spark.streaming.StreamingContext._ import org.apache.spark.api.java.function._ import org.apache.spark.streaming.api._ import org.apache.spark.storage.StorageLevel val ssc = new StreamingContext(sc, Seconds(5)) // read from text file val lines0 = ssc.textFileStream("test") val words0 = lines0.flatMap(_.split(" ")) // read from socket val lines1 = ssc.socketTextStream("localhost", 9999, StorageLevel.MEMORY_AND_DISK_SER) val words1 = lines1.flatMap(_.split(" ")) val words = words0.union(words1) val wordCounts = words.map((_, 1)).reduceByKeyAndWindow(_ + _, Seconds(30), Seconds(10)) wordCounts.print() ssc.checkpoint(".") ssc.start() ssc.awaitTermination()  However, I am getting error from this line: val wordCounts = words.map((_, 1)).reduceByKeyAndWindow(_ + _, Seconds(30), Seconds(10))  . Especially, from _ + _. The error is 51: error: missing parameter type for expanded function ((x$2, x$3) => x$2.$plus(x$3))


Could anybody tell me what the problem is? Thanks!

How to fix the pattern-matching exhaustive warning?

Some scala code:

val list = List(Some("aaa"), Some("bbb"), None, ...)

list.filter(_!=None).map {
case Some(x) => x + "!!!"
// I don't want to handle None case since they are not possible
// case None
}


When I run it, the compiler complains:

<console>:9: warning: match may not be exhaustive.
It would fail on the following input: None
list.filter(_!=None).map {
^
res0: List[String] = List(aaa!!!, bbb!!!)


How to fix that warning without providing the case None line?

QuantOverflow

Opposite of Tail-Risk Hedge (Established Vocabulary)

I'm working on a client memo explaining several approaches to equity hedging, and I'm looking for a not-too-technical term for a hedging strategy where I try to keep options near the money, as to have a quickly reacting hedge, expensive, but drastically reduced drawdown (hopefully).

Of course, the opposite would simply be called tail-risk hedge, but what if I tighten the moneyness? I was thinking about core hedge, near-the-money hedge, continuously adjusted hedge, ...

Obviously this is just a marketing buzzword, but is there some established word that will be understood by most? Also, clients are not too technical, so it may well be a fuzzy word, or not 100% accurate.

StackOverflow

def vs lazy val in case class

I have a DAO object which I defined as a case class.

case class StudentDAO(id: Int) {
def getGPA: Double = // Expensive database lookup goes here
def getRank: Int = // Another expensive database operation and computation goes here
def getScoreCard: File = // Expensive file lookup goes here
}


I would naturally make getGPA and getRank and getScoreCard defs and not vals because I don't want them to be computed before they may be used.

What would be the performance impact if I marked these methods as lazy vals instead of defs? The reason I want to make them lazy vals is: I do not want to recompute the rank each time for a Student with id "i".

I am hoping that this will not be marked as duplicate because there are several questions as below which are mostly about differences:

When to use val, def, and lazy val in Scala?

def or val or lazy val for grammar rules?

def vs val vs lazy val evaluation in Scala

Scala Lazy Val Question

This question is mainly aimed towards the expenses (tradeoffs between CPU vs. memory) in making a method a lazy val for costly operations and what would one suggest over other and why?

EDIT: Thank you for the comment @om-nom-nom. I should have been more clear with what I was looking for.

Use of lazy val for caching string representation

that string representation of the object is cached (see @Dave Griffith's answer). More precisely I am looking at the impact of Garbage Collection if I made it a lazy val instead of def

How to ensure tail recursion consistently

I'd like to make some function be optimized for tail-recursion. The function would emit stackoverflow exception without optimization.

Sample code:

import scala.util.Try
import scala.annotation.tailrec

object Main {
val trials = 10

@tailrec
val gcd : (Int, Int) => Int = {
case (a,b) if (a == b) => a
case (a,b) if (a > b) => gcd (a-b,b)
case (a,b) if (b > a) => gcd (a, b-a)
}

def main(args : Array[String]) : Unit = {
testTailRec()
}

def testTailRec() {
val outputs : List[Boolean] = Range(0, trials).toList.map(_ + 6000) map { x =>
Try( gcd(x, 1) ).toOption.isDefined
}
outputTestResult(outputs)
}

def outputTestResult(source : List[Boolean]) = {
val failed = source.count(_ == false)
val initial = source.takeWhile(_ == false).length
println( s"totally $failed failures,$initial of which at the beginning")
}
}


Running it will produce the following output:

[info] Running Main
[info] totally 2 failures, 2 of which at the beginning


So, first two runs are performed without optimization and are dropped half-way due to the stackoveflow exception, and only later invocations produce desired result.

There is a workaround: you need to warm up the function with fake runs before actually utilizing it. But it seems clumsy and highly inconvenient. Are there any other means to ensure my recursive function would be optimized for tail recursion before it runs for first time?

update:

I was told to use two-step definition

@tailrec
def gcd_worker(a: Int, b: Int): Int = {
if (a == b) a
else if (a > b) gcd(a-b,b)
else gcd(a, b-a)
}
val gcd : (Int,Int) => Int = gcd_worker(_,_)


I prefer to keep clean functional-style definition if it is possible.

AWS

Big Data Update - New Blog and New Web-Based Training

The topic of big data comes up almost every time I meet with current or potential AWS customers. They want to store, process, and extract meaning from data sets that seemingly grow in size with every meeting.

In order to help our customers to understand the full spectrum of AWS resources that are available for use on their big data problems, we are introducing two new resources -- a new AWS Big Data Blog and web-based training on Big Data Technology Fundamentals.

AWS Big Data Blog
The AWS Big Data Blog is a way for data scientists and developers to learn big data best practices, discover which managed AWS Big Data services are the best fit for their use case, and help them get started on AWS big data services. Our goal is to make this the hub for developers to discover new ways to collect, store, clean, process, and visualize data at any scale.

Readers will find short tutorials with code samples, case studies that demonstrate the unique benefits of doing big data on AWS, new feature announcements, partner- and customer-generated demos and tutorials, and tips and best practices for using AWS big data services.

The first two posts on the blog show you how to Build a Recommender witrh Apache Mahout on Amazon Elastic MapReduce and how to Power Gaming Applications with Amazon DynamoDB.

Big Data Training
If you are looking for a structured way to learn more about the tools, techniques, and options available to you as you learn more about big data, our new web-based Big Data Technology Fundamentals course should be of interest to you.

You should plan to spend about three hours going through this course. You will first learn how to identify common tools and technologies that can be used to create big data solutions. Then you will gain an understanding of the MapReduce framework, including the map, shuffle and sort, and reduce components. Finally, you will learn how to use the Pig and Hive programming frameworks to analyze and query large amounts of data.

You will need a working knowledge of programming in Java, C#, or a similar language in order to fully benefit from this training course.

The web-based course is offered at no charge, and can be used on its own or to prepare for our instructor-led Big Data on AWS course.

-- Jeff;

StackOverflow

Is it possible to insert rcords in lobos migration plugin for clojure?

In clojure, I am using lobos library with "mysql/mysql-connector-java" dependency.
I've created some master tables using lobos migration library in clojure.
Now I need to insert some of the default values through the migration itself.
Is there any way the library provide or should I use any other library to write DB migration.

CompsciOverflow

Find an MST when $w: E \to \{ 1,2\}$

I've been asked the following question:

Given a connected undirected graph $G=(V,E)$ and a weight function $w: E \to \{1,2\}$, suggest an efficient algorithm that finds an MST of the graph.

After a few clarifications, the algorithm should run in time $O(|V|+|E|)$. I believe we can do this because of the fact that there are two types of weight of edges.

I tried the following idea, which did not work (I had a contradicting example):

Run BFS on the graph, and output the BFS tree.

I thought that shortest paths in this case would also mean easiest trees, but this is not the case.

I also tried to make each edge $e$ such that $w(e)=2$ into two edges of weight $1$ but my friend also gave a contradicting example.

Can you help please? This is not a homework question, this is a previous exam question as I am studying for my algorithms exam which is soon.

StackOverflow

sbt subproject cannot find it's dependencies

I have a project tree consisting of three projects A, B and C

B depends on A, and C depends on both A and B.

A and B are checked out in C's lib/ and both build fine using sbt compile

However, when I compile C, the build of B fails, complaining that it cannot find certain types/packages:

import org.scalatra.sbt._
import sbt.Keys._
import sbt._

object NwbApiBuild extends Build {
val Organization = "org.nwb"
val Name = "NWB API"
val Version = "0.1.0-SNAPSHOT"
val ScalaVersion = "2.10.3"
val ScalatraVersion = "2.3.0"

lazy val active_slick= Project (
"active-slick",
base = file("lib/active-slick")
)

lazy val slick_auth= Project (
"slick-auth",
base = file("lib/slick-auth")
)

lazy val project = Project (
"root",
file("."),
settings = Defaults.defaultSettings ++ ScalatraPlugin.scalatraWithJRebel ++ Seq(
organization := Organization,
name := Name,
version := Version,
scalaVersion := ScalaVersion,
resolvers += Classpaths.typesafeReleases,
libraryDependencies ++= Seq(
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"ch.qos.logback" % "logback-classic" % "1.0.6" % "runtime",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.8.v20121106" % "container",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" % "container;provided;test" artifacts (Artifact("javax.servlet", "jar", "jar")),
"com.typesafe.slick" %% "slick" % "2.0.2",
"mysql" % "mysql-connector-java" % "5.1.31",
"joda-time" % "joda-time" % "2.3",
"org.joda" % "joda-convert" % "1.5",
"com.github.tototoshi" %% "slick-joda-mapper" % "1.1.0",
"org.json4s" %% "json4s-native" % "3.2.10",
"org.json4s"   %% "json4s-jackson" % "3.2.7",
"c3p0" % "c3p0" % "0.9.1.2"
)
)
) aggregate(active_slick, slick_auth) dependsOn(active_slick, slick_auth)
}


where slick auth has build file

import org.scalatra.sbt._

name := "slick-auth"

version := "0.0.1-SNAPSHOT"

scalaVersion := "2.10.3"

val ScalatraVersion = "2.3.0"

lazy val active_slick = Project(
"active-slick",
base = file("lib/active-slick")
)

lazy val root = Project(
"root",
file("."),
settings = Defaults.defaultSettings ++ ScalatraPlugin.scalatraSettings ++ Seq(
libraryDependencies ++= Seq(
"com.typesafe.slick" %% "slick" % "2.0.2",
"org.slf4j" % "slf4j-nop" % "1.6.4",
"org.scalatest" %% "scalatest" % "2.2.0" % "test",
"org.scalatra" %% "scalatra" % ScalatraVersion,
"org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
"ch.qos.logback" % "logback-classic" % "1.0.6" % "runtime",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.8.v20121106" % "container",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" % "container;provided;test" artifacts (Artifact("javax.servlet", "jar", "jar")),
"com.typesafe.slick" %% "slick" % "2.0.2",
"joda-time" % "joda-time" % "2.3",
"org.joda" % "joda-convert" % "1.5",
"com.github.tototoshi" %% "slick-joda-mapper" % "1.1.0",
"org.json4s" %% "json4s-native" % "3.2.10",
"org.json4s"   %% "json4s-jackson" % "3.2.7",
"c3p0" % "c3p0" % "0.9.1.2"
)
)
).aggregate(active_slick).dependsOn(active_slick)


and active_slick:

name := "active-slick"

version := "0.0.1-SNAPSHOT"

scalaVersion := "2.10.3"

libraryDependencies ++= Seq(
"com.typesafe.slick" %% "slick" % "2.0.2",
"org.slf4j" % "slf4j-nop" % "1.6.4",
"org.scalatest" %% "scalatest" % "2.2.0" % "test",
"com.h2database" % "h2" % "1.3.166" % "test"
)


StackOverflow

Rendering fields in Play Framework 2 Form when editing existing object

This may seem a pretty basic question. I'm using Play Framework 2.1.5 with Scala 2.10 to develop a CRUD app (it will have more in the future). I'm also using Twitter Bootstrap 3.2.0. I'm not able to get my template to render the values for a form when I am attempting to update an object.

I'm following along the book Play for Scala. I can already create a new object. Now I want to be able to edit the object.

The Product domain object is mapped using Squeryl:

case class Product(ean: Long, name: String, description: String) extends KeyedEntity[Long]


The companion Person object has the appropriate CRUD functionality. Product.findByEan(ean) is a simple search by field query that returns an Option[Person].

The Persons controller has the Form[Person] mapping:

private val productForm = Form(mapping(
"ean" -> longNumber.verifying("validation.ean.duplicate", Product.findByEan(_).isEmpty),
"name" -> nonEmptyText,
"description" -> nonEmptyText)(Product.apply)(Product.unapply))


In my routes

GET        /products                         controllers.Products.list
POST       /products                         controllers.Products.save
GET        /products/new                     controllers.Products.newProduct
GET        /products/$ean<\d{13}> controllers.Products.show(ean: Long) GET /products/$ean<\d{13}>/edit       controllers.Products.edit(ean: Long)


The Products.edit route ought to load the Product using its ean field and prefill out the edit form with the found product.

def edit(ean: Long) = Action { implicit request =>
Product.findByEan(ean) match {
case Some(product) =>
Logger.info("found product: ean = " + ean)
val form = productForm.fill(product)
Logger.info(form.toString)
Ok(views.html.products.editProduct(form))
case None => Redirect(routes.Products.list)
}
}


I know for a fact that I'm successfully retrieving the object as the Logger messages clearly show this in the play console.

The editProducts template to render the form is:

@(productForm: Form[Product])(implicit flash: Flash, lang: Lang)
@import helper._

@productsLayout(Messages("products.form")) {
<h2>@Messages("products.form")</h2>

@helper.form(action = routes.Products.save()) {
<fieldset>
<legend>
@Messages("products.details", Messages("products.edit"))
</legend>
@helper.inputText(productForm("product.ean"))
@helper.inputText(productForm("product.name"))
@helper.textarea(productForm("product.description"))
</fieldset>
<p><input type="submit" class="btn primary" value='@Messages("products.edit.submit")'</p>
}
}


The problem is that that when I attempt to edit my Product object, this form template is not pre-filling out the <input> tag with the value attribute from the retrieved product object. The same template works just fine when I try to create an object but submit a form with validation errors.

1. Has anyone else experienced this issue?
2. Does the twitterBootstrap helper assume Bootstrap 2 rather than Bootstrap 3?

Ben Horowitz

How to Ruin Your Company with One Bad Process

“Real quick, whole squad on that real sh*t
0 to 100, n***a, real quick”
—Drake, "0 to 100/The Catch Up"

I am a giant advocate for technical founders running their own companies, but one consistent way that technical founders deeply harm their businesses is by screwing up the budgeting process. Yes, the budgeting process. How ridiculous is that? How does it happen and why is it particularly problematic for engineers?

I'll begin by describing how I screwed it up in my company. Our sales were growing so fast that the biggest problem that we faced was that we literally could not handle all the customers that wanted to sign up for Loudcloud. To combat this and enable us to grow, I worked diligently with my team to plan all the activities that we needed to accomplish to expand our capacity and capture the market before the competition. Next, I assigned sub-goals and activities to each functional head. In conjunction with my leadership team, I made sure that each goal was measurable and supported by paired metrics as well as lagging and leading indicators. I then told the team to figure out what it would take to accomplish those goals and return with their requirements for headcount and program dollars. Finally, I made adjustments to their requests based on industry benchmarks (mostly reductions) to get to a plan that I thought made sense.

Here’s the basic process:

1. Set goals that will enable us to grow
2. Break the goals down so that there is clear ownership and accountability for each goal by a specific team
3. Refine goals into measurable targets
4. Figure out how many new people are required to hit the targets
5. Estimate the cost of the effort
6. Benchmark against the industry
7. Make global optimizations
8. Execute

Unless you are an experienced manager, you may not even see what’s wrong with this process, but it very nearly led to my company's demise. In fact, the above process is completely upside-down and should only be followed if you wish to bloat your company to the brink of bankruptcy and create a culture of chaos.

When I asked my managers what they needed, I unknowingly gamified the budgeting process. The game worked as follows: The objective was for each manager to build the largest organization possible and thereby expand the importance of his function. Through the transitive property of status, he could increase his own importance as well. Now you may be thinking, "That wouldn't happen in my company. Most of my staff would never play that game." Well, that's the beauty of the game. It only takes one player to opt in, because once someone starts playing, everybody is going in -- and they are going in hard.

Gameplay quickly becomes sophisticated as managers develop clever strategies and tactics to improve their chances for winning. One common game technique is to dramatically expand the scope of the goals: "When you said that you wanted to increase our market presence, I naturally assumed that you meant globally. Surely, you wouldn’t want me to take a U.S.-centric view." To really motivate the CEO, another great technique involves claiming dire circumstances if the company fails to achieve its metrics: "If we don't increase sales by 500% and our top competitor does, we will fall behind. If we fall behind, we will no longer be No. 1. If we’re not No. 1, then we won’t be able to hire the best people, command the best prices, or build the best product, and we will spin into a death spiral." Never mind the fact that there is almost no chance that your competitor will grow 500% this year.

Another subtle problem with this process is that when I asked my team what they needed to achieve their goals, they naturally assumed they would get it. As a result, my team deeply socialized their ideas and newly found money with their teams. This has the added gaming benefit of inextricably tying their demands to company morale. When the VP of marketing asked me for 10 headcount and $5M in program expenses, then shared that plan with his team, it changed the conversation. Now a major cutback to his plan would alarm his team because they had just spent two weeks planning for a much more positive scenario. “Wow, Ben greatly reduced the plan. Should I be looking for a job?” This kind of dynamic put pressure on me to create a more expansive expense plan than was wise. Multiply this by all my managers and I was on my way to burning up all my cash and destroying my culture. My core problem was that my budgeting process did not have any real constraints. We were private and did not have a specific profit target that we needed to hit and we had plenty of cash in the bank. Drawing the line on expenses seemed rather arbitrary. In the absence of a hard constraint, I had no constraint. An excellent constraining principle when planning your budget is the preservation of cultural cohesion. The enemy of cultural cohesion is super-fast headcount growth. Companies that grow faster than doubling their headcount annually tend to have serious cultural drift, even if they do a great job of onboarding new employees and training them. Sometimes this kind of growth is necessary and manageable in certain functions like sales, but is usually counterproductive in other areas where internal communication is critical like engineering and marketing. If you quadruple your engineering headcount in a year, you will likely have less absolute throughput than if you doubled headcount. As an added bonus, you will burn way more cash. Even worse, you will lose cultural consistency as new people with little guidance will come in with their own way of doing things that doesn’t match your way of doing things. Note that this does not apply to you if you have very small numbers. It's fine to grow engineering from one to four people or from two to eight. However, if you try to grow from 50 to 200, you will cause major issues if you are not extremely careful. Starting with the cultural cohesion principle, a far better way to run the budgeting process is to start with the constraints. Some useful constraints are: • Run rate increase – Note that I say "run rate increase" and not "spend increase". You should set a limit on the amount by which you are willing to increase what you are spending in the last month of the coming year vs. the previous year. • Earnings/Loss – If you have revenue, another great constraint is your targeted earnings or loss for the year. • Engineering growth rate – Unless you are making an acquisition and running it separately or sub-dividing engineering in some novel way, you should strive not to more than double a monolithic engineering organization in a 12-month period. • Ratio of engineering to other functions – Once you have constrained engineering, then you can set ratios between engineering and other functions to constrain them as well. After applying the global constraints, the following steps will provide a better process: 1. Take the constrained number that you created and reduce it by 10-25% to give yourself room for expansion, if necessary. 2. Divide the budget created above in the ratios that you believe are appropriate across the team. 3. Communicate the budgets to the team. 4. Run your goal-setting exercise and encourage your managers to demonstrate their skill by achieving great things within their budgets. 5. If you believe that more can legitimately be achieved in a group with more money, then allocate that manager extra budget out of the slush fund you created with the 10-25%. At this point, some readers may think that I've lost my mind. As a technologist, you know that the worst thing that you can do is over-constrain the problem before you start. You'll kill creativity and prevent yourself from getting a truly great outcome. That's precisely why I, as an engineer, struggled with this process: the human factors muck up the logic. Specifically, local incentives, if not properly managed, will sharply motivate human behavior and defeat the global goals. It’s critical to recognize this so that you don’t turn your agile, small company into a slow, big company before its time. QuantOverflow What are the implication of a negative risk-free rate on SML? What happens to the Security market line (within the CAPM model) when the risk-free rate turns negative? StackOverflow PartialFunction with recursion [duplicate] This question already has an answer here: I have written this code sample: val fact: PartialFunction[Int, Int] = { case 0 => 1 case n if n > 0 => n * fact(n-1) }  And it fails to compile with an error: forward reference extends over definition of value fact. So my question is whether a partial function can be recursive? May be there is another syntax for that? EDIT This error seems to appear only when trying this in Scala Worksheet. /r/netsec Phishing Frenzy integrating BeEF for HTA PowerShell Attacks Lobsters ELF Research Projects Some cool ELF manipulation research projects. Comments /r/netsec My MiniLock Concerns TheoryOverflow Known time complexity advantage of quantum algorithms over classical algorithms I know that this question may depend on how one formulates each complexity class, but in general, what time complexity advantage does quantum algorithms have over classical algorithms? Definition of Planar 3-SAT What is the exact definition of Planar 3-SAT? And how can I find a good reference for it, and also for the proof of it's NP-completeness? I need the precise definition? I am a little confused by the results that I found in Google! Best regards, Elham Planet Emacsen Emacs Redux: Travel back and forward in git history I recently discovered an extremely cool package called git-timemachine that allows you to step though the git history of the file you’re currently editing in Emacs. Using it is pretty simple: • visit a git-controlled file and issue M-x git-timemachine (or bind it to a keybinding of your choice) • use the following keys to navigate historic version of the file: • p visit previous historic version • n visit next historic version • w copy the hash of the current historic version • q exit the time machine buffer Here you can see git-timemachine in action: P.S. This package is bundled with Prelude. High Scalability Sponsored Post: Apple, Asana, FoundationDB, Cie Games, BattleCry, Surge, Dreambox, Chartbeat, Monitis, Netflix, Salesforce, Cloudant, CopperEgg, Logentries, Gengo, Couchbase, MongoDB, BlueStripe, AiScaler, Aerospike, LogicMonitor, AppDynamics, ManageEngine, Site24x7 Who's Hiring? • Apple has multiple openings. Changing the world is all in a day's work at Apple. Imagine what you could do here. • Software Developer in Test. The iOS Systems team is looking for a Quality Assurance engineer. In this role you will be expected to work hand-in-hand with the software engineering team to find and diagnose software defects. The ideal candidate will also seek out ways to further automate all aspects of our existing process. This is a highly technical role and requires in-depth knowledge of both white-box and black-box testing methodologies. Please apply here • Senior Software Engineer -iOS Systems. Do you love building highly scalable, distributed web applications? Does the idea of a fast-paced environment make your heart leap? Do you want your technical abilities to be challenged every day, and for your work to make a difference in the lives of millions of people? If so, the iOS Systems Carrier Services team is looking for a talented software engineer who is not afraid to share knowledge, think outside the box, and question assumptions. Please apply here. • Software Engineering Manager, IS&T WWDR Dev Systems. The WWDR development team is seeking a hands-on engineering manager with a passion for building large-scale, high-performance applications. The successful candidate will be collaborating with Worldwide Developer Relations (WWDR) and various engineering teams throughout Apple. You will lead a team of talented engineers to define and build large-scale web services and applications. Please apply here. • C++ Senior Developer and Architect- Maps. The Maps Team is looking for a senior developer and architect to support and grow some of the core backend services that support Apple Map's Front End Services. Ideal candidate would have experience with system architecture, as well as the design, implementation, and testing of individual components but also be comfortable with multiple scripting languages. Please apply here. • Cie Games, small indie developer and publisher in LA, is looking for rock star Senior Game Java programmers to join our team! We need devs with extensive experience building scalable server-side code for games or commercial-quality applications that are rich in functionality. We offer competitive comp, great benefits, interesting projects, and exceptional growth opportunities. Check us out at http://www.ciegames.com/careers. • BattleCry, the newest ZeniMax studio in Austin, is seeking a qualified Front End Web Engineer to help create and maintain our web presence for AAA online games. This includes the game accounts web site, enhancing the studio website, our web and mobile- based storefront, and front end for support tools. http://jobs.zenimax.com/requisitions/view/540 • FoundationDB is seeking outstanding developers to join our growing team and help us build the next generation of transactional database technology. You will work with a team of exceptional engineers with backgrounds from top CS programs and successful startups. We don’t just write software. We build our own simulations, test tools, and even languages to write better software. We are well-funded, offer competitive salaries and option grants. Interested? You can learn more here. • Asana. As an infrastructure engineer you will be designing software to process, query, search, analyze, and store data for applications that are continually growing in scale. You will work with a world-class team of engineers on deploying and operating existing systems, and building new ones for problems that are unique to our problem space. Please apply here. • Operations Engineer - AWS Cloud. Want to grow and extend a cutting-edge cloud deployment? Take charge of an innovative 24x7 web service infrastructure on the AWS Cloud? Join DreamBox Learning’s creative team of engineers, designers, and educators. Help us radically change education in an environment that values collaboration, innovation, integrity and fun. Please apply here. http://www.dreambox.com/careers • Chartbeat measures and monetizes attention on the web. Our traffic numbers are growing, and so is our list of product and feature ideas. That means we need you, and all your unparalleled backend engineer knowledge to help up us scale, extend, and evolve our infrastructure to handle it all. If you've these chops: www.chartbeat.com/jobs/be, come join the team! • The Salesforce.com Core Application Performance team is seeking talented and experienced software engineers to focus on system reliability and performance, developing solutions for our multi-tenant, on-demand cloud computing system. Ideal candidate is an experienced Java developer, likes solving real-world performance and scalability challenges and building new monitoring and analysis solutions to make our site more reliable, scalable and responsive. Please apply here. • Sr. Software Engineer - Distributed Systems. Membership platform is at the heart of Netflix product, supporting functions like customer identity, personalized profiles, experimentation, and more. Are you someone who loves to dig into data structure optimization, parallel execution, smart throttling and graceful degradation, SYN and accept queue configuration, and the like? Is the availability vs consistency tradeoff in a distributed system too obvious to you? Do you have an opinion about asynchronous execution and distributed co-ordination? Come join us • Human Translation Platform Gengo Seeks Sr. DevOps Engineer. Build an infrastructure capable of handling billions of translation jobs, worked on by tens of thousands of qualified translators. If you love playing with Amazon’s AWS, understand the challenges behind release-engineering, and get a kick out of analyzing log data for performance bottlenecks, please apply here. • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here. • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here. Fun and Informative Events • OmniTI has a reputation for scalable web applications and architectures, but we still lean on our friends and peers to see how things can be done better. Surge started as the brainchild of our employees wanting to bring the best and brightest in Web Operations to our own backyard. Now in its fifth year, Surge has become the conference on scalability and performance. Early Bird rate in effect until 7/24! Cool Products and Services • A third party vendor benchmarked the performance of three databases: Couchbase Server, MongoDB and DataStax Enterprise. The databases were benchmarked with two different workloads (read-intensive / balanced) via YCSB on dedicated servers. Read. • Now track your log activities with Log Monitor and be on the safe side! Monitor any type of log file and proactively define potential issues that could hurt your business' performance. Detect your log changes for: Error messages, Server connection failures, DNS errors, Potential malicious activity, and much more. Improve your systems and behaviour with Log Monitor. • The NoSQL "Family Tree" from Cloudant explains the NoSQL product landscape using an infographic. The highlights: NoSQL arose from "Big Data" (before it was called "Big Data"); NoSQL is not "One Size Fits All"; Vendor-driven versus Community-driven NoSQL. Create a free Cloudant account and start the NoSQL goodness • Finally, log management and analytics can be easy, accessible across your team, and provide deep insights into data that matters across the business - from development, to operations, to business analytics. Create your free Logentries account here. • CopperEgg. Simple, Affordable Cloud Monitoring. CopperEgg gives you instant visibility into all of your cloud-hosted servers and applications. Cloud monitoring has never been so easy: lightweight, elastic monitoring; root cause analysis; data visualization; smart alerts. Get Started Now. • Whitepaper Clarifies ACID Support in Aerospike. In our latest whitepaper, author and Aerospike VP of Engineering & Operations, Srini Srinivasan, defines ACID support in Aerospike, and explains how Aerospike maintains high consistency by using techniques to reduce the possibility of partitions. • LogicMonitor is the cloud-based IT performance monitoring solution that enables companies to easily and cost-effectively monitor their entire IT infrastructure stack – storage, servers, networks, applications, virtualization, and websites – from the cloud. No firewall changes needed - start monitoring in only 15 minutes utilizing customized dashboards, trending graphs & alerting. • BlueStripe FactFinder Express is the ultimate tool for server monitoring and solving performance problems. Monitor URL response times and see if the problem is the application, a back-end call, a disk, or OS resources. • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Cloud deployable. Free instant trial, no sign-up required. http://aiscaler.com/ • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications. • www.site24x7.com : Monitor End User Experience from a global monitoring network. If any of these items interest you there's a full description of each sponsor below. Please click to read more... DataTau Show DT: A crash course in A/B testing (SlideShare) TheoryOverflow NEXPTIME-completeness with more time for reductions One thing that surprised me when learning about complexity theory is that for a complexity class C, we tend to define C-complete using polynomial time reductions, even when C is a very large complexity class. This definitely makes sense for P and NP, but how about EXPTIME and NEXPTIME? So my question is this: What is the subset of NEXPTIME comprised of problems exp-time reducible from every other problem in NEXPTIME? If EXPTIME != NEXPTIME, is this class distinct from both NEXPTIME and NEXPTIME-complete? /r/netsec Another Network Forensics Puzzle Lobsters Kafka Producer Pipeline for Ruby on Rails A Tour of Transit This tour introduces the Transit format via transit-js, an implementation of the Transit format for JavaScript. Transit is a format and set of libraries for conveying values between applications written in different programming languages. Comments /r/clojure Transit Planet Theory The Burden of Large Enrollments This week I'm at the CRA Snowbird conference, the biennial meeting of CS chairs and other leaders in the field. In 2012 many of the discussion focused on MOOCS. This year the challenge facing most CS chairs are booming enrollments in CS courses. A nice problem to have, but a problem nevertheless. Last night we had a broad discussion about the burgeoning number of students. Ed Lazowska showed his NCWIT slides giving anecdotal evidence. It's too early to get a complete survey of CS departments but hardly anyone in the audience felt that enrollments were not going up by double (or triple) digit percentages. Not only an increase in majors, but a number of non-majors, minors or others, who take upper level courses. At Georgia Tech it seems every engineering student wants to minor in CS. We all have to deal with the increase in the short term. Depending on the institution, we have more classes, larger classes, enrollment caps, increases in instructors, PhD lecturers, teaching postdocs, automated grading, undergrad and MS TAs and having other departments take up some of the load. All of this could hurt the undergrad experience but with careful planning we can mitigate those effects. What's driving the increase? Some suggested a change in the ubiquitous view of computing and the more positive opinion of geeks in society. More likely though, the driver is jobs, the great need for computer scientists and not just from computing companies, and the limited jobs for those graduating with non-technical majors. Is the big shifts in enrollment a permanent change or just another part of the boom and bust cycle in CS? More than a theoretical questions, a permanent change makes for a better argument with deans and provosts to increase faculty sizes in computer science. There are some signs of "this time is different," such as the increase in non-majors in upper level courses, but it's an argument that will have a hard sell unless the increase is sustained for several years. One good argument: If you draw a linear line through enrollments since the 70's you get a consistent 10% yearly increase averaged over booms and busts. Lobsters Just A Regular Night with WindStream DSL StackOverflow How do you install an external Java library such as caldav4j in my leiningen managed app? Goes without saying that I'm new to clojure. As a learning project, I'd like to use the caldav4j library to pull down work calendar events however I don't know how to properly pull in an external library that is not found via "lein search". Update: Sharing my working project.clj file for reference: (defproject caldav-sync "0.1.0-SNAPSHOT" :description "Extract and synchronize calendar entries locally" :url "http://example.com/FIXME" :license {:name "Eclipse Public License" :url "http://www.eclipse.org/legal/epl-v10.html"} :dependencies [[org.clojure/clojure "1.6.0"] [org.osaf/caldav4j "0.7"]] :repositories [["caldav4j" "https://caldav4j.googlecode.com/svn/maven/"]])  /r/clojure ClojureScript debugging tips? I'm having a fun time coding a small single-page app with Om and core.async, but I'm having trouble getting insight into my code when it goes wrong. With source maps, I can view the entire ClojureScript call stack at a breakpoint, but I can't inspect the values at each level of the call stack. Viewing the JS values is sometimes useful, but frequently my execution is paused deep in the ClojureScript internals, and it's difficult to figure out what JS values are directly relevant to the ClojureScript in question. With the browser REPL, I can execute arbitrary ClojureScript, including functions in my current namespace -- but it is hard to recreate the entire execution context of my bug. In Om, for instance, it's hard to debug a component, because it relies on an app argument which is actually a short-lived cursor created from my app state. (I run into a similar problem when trying to debug Ring wrappers; how do I generate a full-featured Ring request/response while playing in the REPL?) My goal is to run the actual code, pause at a breakpoint, and then inspect the ClojureScript values in play. Is this possible with the current technology? Currently, I'm just using a bunch of (.log js/console "Doing something with %o [%s]" obj (pr-str obj)) statements... but that's tedious. Right now I feel like I'm going to be slower in CLJS than in JS, even after climbing the learning curve. A stronger debugging story would help a lot. submitted by iron_ball [link] [2 comments] QuantOverflow What is the intuition behind the fact that Modified duration = Macaulay Duration / (1+r)? I understand the derivation of both:take dP/dR and divide by P which will give you both 1) modified duration OR 2) macaulay duration / (1+r) (notice the weighted average time built into the function from taking the derivative - the math makes sense) My question is about intuition: how can discounting the weighted average time to maturity by an extra period be equal to the sensitivity of %price to %yield? Perhaps using continuously compounded returns can help in the intuition? Should I memorize the fact and move on and not require intuition behind it TheoryOverflow How can you prove that all halting probabilites are normal real numbers? Wikipedia claims that any halting probability (Chaitin's constant) is a normal number. Since Chaitin's constant is uncomputble, how is a proof the the normalcy of the number possible? Computable numbers like pi and e are thought to be normal, yet there is no proof. It seems surprising to me that we have a proof of Chaitin's constant being normal although it is an uncomputable number, especially since "Chaitin's constant" isn't actually one number, but an infinite set of numbers (since it depends on the program encoding used). CompsciOverflow Search in a partial ordering defined by tuples of numbers This is a graph theory and partial ordering problem. Consider a set of triples {(di,ai,ci)}i=1...N, which specify edges between two nodes A and B, d denotes a departure time, a an arrival time and c a cost. Technically there can be multiple incomparable costs, for example in$ and % chance that your goods arrive safely.

An example of such a situation could be these 5 edges:

(I did not draw the time axis on scale)

There are five edges, I, II, III, IV and V. The costs are denoted at the edges, they are either 100, 10 or 1. Edge V is drawn red to easily discriminate it from the other edges, since it crosses through them. However, aside from that it is not different.

Given such edges, a few things are important:

• An edge is only interesting if it departs after we arrive at A, for example in the image we can arrive at (a), then edge I is not an option anymore. The same goes for edges II and V if we arrive at (b), etc.
• Given an arrival and its set of interesting edges, and edge ei is dominated if there exists an edge ej with ej< ei. This is the case iff aj ≤ ai ^ cj ≤ ci ^ (aj < ai v cj < ci). In layman terms, cost and arrival need to be at least as small, but one must be strictly smaller. This is a partial ordering, some edges are incomparable.
• Given a arrival in A, I want to find all relevant, undominated (or Pareto) edges.

We can enumerate all the edges:

• I: (0 , 1, 100)
• II: (0.5, 2, 10)
• III: (2 , 3, 100)
• IV: (2.5, 4, 10)
• V: (1.5, 5, 1)

Putting them in a partial ordering, using a directed acyclic graph (dag), we get this:

An arrow denotes the <-relation between edges. Note that an edge in this case is a node in the dag: confusing, I know. When I say edge, I mean a node in the dag above.

I added an edge (-∞, -∞), which is always irrelevant, but creates a nice 'root' of the dag. This way we sort of have a tree, where leaves sometimes merge without creating cycles. I also only denote the arrival at B and cost, needed to create the dag, but technically there is also a departure are A for each of the edges.

If we were to arrive at A at -1, we can simply return all the children of (-∞, -∞) as relevant and undominated, but for another arrival we may need to traverse the tree differently (in a more complicated fashion). I was thinking of this:

marker := map from edge to bool # marks whether edge has been traversed
for all edges: marker(edge) := false # none have been traversed

traverse(root, arrival):
if(marker(root)) return []  #already been at this node
marker(root) := true;
if(departure(root) > arrival) return [root]; # return singleton list, all children will be dominated by this edge.

# this node is irrelevant, but possible some children are relevant.
l = [];
for each child of root:
append traverse(child, arrival) to l
return l


Traverse returns a list of undominated, relevant edges. However, Is this an efficient way to tackle this problem? Is there a better solution to this problem? Is this problem known under a known name?

StackOverflow

Select no fields from table in Korma

I'm trying to do a join across a number of tables (three plus a join table in the middle). I think korma is lazily evaluating the last join. What I'm trying to do is to add a condition that restricts the results of the first table in the join, but I'm only interested in fields from the last table in the join.

So for example say I've got clubs, people and hobbies tables, and a people-to-hobbies join table for the last two. Each club can have many people, and each person can have many hobbies.

I'm trying to get the full details of all the hobbies of people who belong to a specific club, but I don't want any fields from the club table. The join table means that korma will create two queries, one to get all the people that are in a specific club, and another to retrieve the hobbies for that person via the people-to-hobbies join table.

My korma query looks something like this:

(select clubs
(with people
(with hobbies
(fields :hobby-name :id)
(where {:clubs.name "korma coders"}))))


The problem is that I haven't specified which fields I want from clubs and people, and the default is to select *. How can I include no fields from these tables? Is this possible, or does the fact that the hobbies are lazily loaded mean korma has to return some results in the first query (which gets me a filtered list of people), so that when I come to interrogate it later for the hobbies, it has the ids it needs to run the second query?

Planet Emacsen

Emacs Redux: Become an Emacs Redux patron

Securing funding for OS projects and blogs (outside of displaying annoying ads) is a hard endeavor. While I’d love to raise enough cash to work on CIDER, projectile and Emacs itself for an entire year through some wildly successful crowdfunding campaign, that’s unlikely to ever happen.

That said, I recently discovered an interesting alternative to one-off crowdfunding campaigns called gittip. Self-described as sustainable crowdfunding (meaning people will get microdonations (say half a buck) from their patrons each week) it seems like a reasonable way to raise a modest sum of money so you could work on open-source projects more than your regular schedule might permit.

So, if you like Emacs Redux and the Emacs projects in which I’m involved like:

• Prelude - An Emacs distribution built on top of GNU Emacs 24
• Projectile - Project Interaction Library for Emacs, that stays out of your way
• clojure-mode - A major mode for programming in Clojure
• cider - A Clojure programming environment for Emacs
• guru-mode - An annoying companion on your journey to Emacs mastery
• rubocop-emacs - Emacs integration for RuboCop
• zenburn-emacs - The Zenburn color theme, ported to Emacs
• solarized-emacs - The Solarized color theme, ported to Emacs
• puppet-mode - A mode for editing Puppet 3 manifests
• swift-mode - A mode for editing Swift code

consider supporting my work via gittip.

CompsciOverflow

Call‑by‑name will succeeds where call‑by‑value may fails: some example cases?

I've landed to SML pages, comparing call‑by‑name and call‑by‑value, asserting the former always succeed while the latter may fails. As this seems counter intuitive to me, I feel at least an example case would be useful.

Call-by-value, call-by-name and call-by-need

Theorem (Church Rosser 1)

For a purely functional language, if call-by-value evaluation and call-by-name evaluation both yield a well-defined result then they yield the same result.

Theorem (Church Rosser 2)

If a well-defined result exists for an expression then the call-by-name evaluation strategy will find it where, in some cases, call-by-value evaluation will not.

What make it counter‑intuitive to me, is that call‑by‑name may re‑evaluate expressions multiple times, which I don't understand how it can make it succeed when call‑by‑value and its one‑time evaluation, would fails. If, as the context is FP and its referential transparency, both expression are supposed to evaluated to the same, how evaluating multiple time may make something succeed or alternatively potentially fails if the same expression is evaluated only once? Unless there are side‑effect somewhere?… is this related to side effects and non‑pure FP?

As a reminder, here is what would look like call‑by‑value and call‑by‑name with SML (if there is an error here, feel free to tell), added call‑by‑need to help not confuse one with the others:

(* This, is supposed to sometime fails… *)

fun call_by_value a =
a + a  (* a is evaluated only once *)

(* …when this is supposed to succeed *)

fun call_by_name a =
a () + a ()  (* a is evaluated for each of its reference *)

(* None of the above should be confused with call‑by‑need: *)

fun call_by_need a =
let val a' = a ()  (* memoization *)
in fn () => a' + a'  (* to be evaluated when needed *)
end

val _ = print (Int.toString (call_by_value 2) ^ "\n")
val _ = print (Int.toString (call_by_name (fn () => 2)) ^ "\n")
val r = call_by_need (fn () => 2) ()  (* memoization *)
val _ = print (Int.toString r ^ "\n")


Fefe

Das Verfahren zu Josef S ist vorbei. Josef S wurde ...

Das Verfahren zu Josef S ist vorbei. Josef S wurde schuldig gesprochen, in allen Anklagepunkten. Er wurde zu zwölf Monaten Haft verurteilt, davon acht bedingt. Die übrigen vier hat er schon in U-Haft abgesessen. So hat sich das Gericht aus der Sache rausgemogelt, ohne eine Schuld einzugestehen oder die Farce dieses Verfahrens zuzugeben. Eine schwere Enttäuschung, dass die Richter nicht die Größe hatten, angesichts dieser lächerlichen Beweislage einen Freispruch auszusprechen.

Das Urteil fußte auf den Aussagen eines einzelnen, sich bei Nachfragen mehrfach selbst widersprechenden, Polizeibeamten. Wieso würde der sich wahllos einen Unschuldigen aus der Menge picken, fragt der Richter. Irgendwas wird schon dran sein.

TheoryOverflow

Examples where the uniqueness of the solution makes it easier to find

The complexity class $\mathsf{UP}$ consists of those $\mathsf{NP}$-problems that can be decided by a polynomial time nondeterministic Turing machine which has at most one accepting computational path. That is, the solution, if any, is unique in this sense. It is thought highly unlikely that all $\mathsf{UP}$-problems are in $\mathsf{P}$, because by the Valiant-Vazirani Theorem this would imply the collapse $\mathsf{NP}=\mathsf{RP}$.

On the other hand, no $\mathsf{UP}$-problem is known to be $\mathsf{NP}$-complete, which suggests that the unique solution requirement still somehow makes them easier.

I am looking for examples, where the uniqueness assumption leads to a faster algorithm.

For example, looking at graph problems, can a maximum clique in a graph be found faster (though possibly still in exponential time), if we know that the graph has a unique maximum clique? How about unique $k$-colorability, unique Hamiltonian path, unique minimum dominating set etc.?

In general, we can define a unique-solution version of any $\mathsf{NP}$-complete problem, scaling them down to $\mathsf{UP}$. Is it known for any of them that adding the uniqueness assumption leads to a faster algorithm? (Allowing that it still remains exponential.)

Planet Clojure

Transit

We are pleased to announce today the initial release of Transit.

Transit is a format and set of libraries for conveying values between applications written in different programming languages. The key objectives of Transit are to support:

• Sending values between applications
• written in different programming languages
• without requiring schemas/context
• i.e., to be self-describing at the bottom
• with extensibility
• and good performance
• with reach to the browser

JSON currently dominates similar use cases, but it has a limited set of types, no extensibility, and is verbose. Actual applications of JSON are rife with ad hoc and context-dependent workarounds for these limitations, yielding coupled and fragile programs.

On the other hand, the reach of JSON is undeniable. High performance parsers are widely available. Thus Transit is specified as an encoding to and from both JSON and MessagePack, a binary JSON-like format with widely available parsers. In particular, both formats have parsers written in C for languages like Ruby and Python that reach to C for performance.

Transit supports a minimal but rich set of core types:

• strings
• booleans
• integers (to 64 bits w/o truncation)
• floats
• nil/null
• arrays
• maps (with arbitrary scalar keys, not just strings)

Transit also includes a wider set of extension types:

• timestamps
• UUIDs
• URIs
• arbitrary precision integers and decimals
• symbols, keywords, characters
• bytes
• sets
• lists
• maps with composite keys

Transit is extensible - users can define extension types in exactly the same way as the included extension types.

The emphasis of Transit is on communication between programs, thus it prioritizes programmatic types and data structures over human readability and document orientation. That said, it does have a readable verbose mode for JSON.

Transit is self describing using tags, and encourages transmitting information using maps, which have named keys/fields which will repeat often in data. These overheads, typical of self-describing formats, are mitigated in Transit by an integrated cache code system, which replaces repetitive data with small codes. This yields not only a reduction in size, but also an increase in performance and memory utilization. Contrast this with gzipping, which, while it may reduce size on the wire, takes time and doesn't reduce the amount of text to be parsed or the number of objects generated in memory after parsing.

We are shipping an 0.8 version of the Transit spec, which has extensive documentation for implementors, as well as interoperable implementations for:

We welcome feedback and suggestions on the transit-format list.

I'd like to thank the team at Cognitect that built Transit:

• Tim Ewald - Team Lead
• Brenton Ashworth
• Timothy Baldridge
• Bobby Calderwood
• David Chelimsky
• Paul deGrandis
• Benoit Fleury
• Michael Fogus
• Ben Kamphaus
• Alex Miller
• David Nolen
• Russ Olsen

I hope you find Transit useful, and look forward to your feedback.

Rich

StackOverflow

"Shared" Connection on Monger Clojure

I am start with monger today, the docs use examples like this:

(let [conn (mg/connect)
db   (mg/get-db conn "monger-test")
coll "documents"]
(mc/insert db coll {:first_name "John"  :last_name "Lennon"})
(mc/insert db coll {:first_name "Ringo" :last_name "Starr"})

(mc/find db coll {:first_name "Ringo"}))


All documentation examples use that structure. Always connect to MongoDB, then use db on mc/insert... . The question is, how I can put this code on a function and use it in my functions that execute mongodb queries, without repeat below code all time:

(let [conn (mg/connect)
db   (mg/get-db conn "monger-test")
coll "documents"] ...


Thanks.

StackOverflow

Scala for Android - Curious type mismatch error

In my Scala Android project I'm getting the curious error:

type mismatch; found : android.accounts.android.accounts.AccountManagerCallback[Boolean] required: android.accounts.android.accounts.AccountManagerCallback[Boolean]

when calling 'removeAccount' in the code below:

val accountManager = android.accounts.AccountManager.get(this)
for (account <- accountManager.getAccounts) {
accountManager.removeAccount(account,
new AccountManagerCallback[Boolean] {
override def run(result: AccountManagerFuture[Boolean]) {
try {
result.getResult(3, TimeUnit.SECONDS)
} catch {
case e: Exception => Log.d(getClass.getName, "exception: ", e)
}
}
}, new Handler())
}
}


Any ideas on what's going wrong and what can I do to placate the type checker?

StackOverflow

What's the appropriate way to use Flask with zeromq in production?

I have a Flask app that accepts HTTP requests. When certain HTTP requests come in, I want to trigger a message on a zeromq stream. I'd like to keep the zeromq stream open all the time. I'm wondering what the appropriate way to do this is. Since it is recommended to use gunicorn with Flask in production, doesn't that mean that there will be multiple instances of the Flask app, and if I put the zeromq connection in the same place as the Flask app, only one of those will be able to connect, and the others will fail.

Clojure : Stuck in writing migration in lobos library, when I am trying to add 2 fields with "timestamp" datatype

In clojure, I am using lobos library with "mysql/mysql-connector-java" dependency.
While writing the migrations to create table, there are 2 fields as "created" and "modified" with the datatype as "timestamp".
When I am not passing any default value for the fields then the library is adding attributes as "on update CURRENT_TIMESTAMP" and default as "CURRENT_TIMESTAMP" for the first field.
i.e. If the "created" will be added first then the default is getting set for it otherwise it is getting set for "modified" field.
I don't want to set the default and ON_UPDATE for created field.
Also the fields are added in some random order in the table.
Can I maintain a sequence to add fields?

Lobsters

Keep It (Really) Stupid Simple!

It’s 2014 a year full with creative web design trends but does that really helps? Minimalist design can be seen at every corner with more or less elements to make us think that is good for us too. However, is not difficult to create such landing page/website? This showcase may answer this question.

StackOverflow

how should I handle polling in ZMQ in node.js?

I'm working on building a Paranoid Pirate in node.js. Starting with some python code:

poller = zmq.Poller()

liveness = HEARTBEAT_LIVENESS
interval = INTERVAL_INIT

heartbeat_at = time.time() + HEARTBEAT_INTERVAL

worker = worker_socket(context, poller)
cycles = 0
while True:
socks = dict(poller.poll(HEARTBEAT_INTERVAL * 1000))

# Handle worker activity on backend
if socks.get(worker) == zmq.POLLIN:
#  Get message
#  - 3-part envelope + content -> request
#  - 1-part HEARTBEAT -> heartbeat
frames = worker.recv_multipart()


The Go sample code also uses a Poller.

The problem that I'm running into is that zeromq.node doesn't seem to use Pollers. Is the solution in node to just use a callback on message? how should I handle the POLLIN state?

Daniel Lemire

Potentially bogus freelancing advice

For nearly 20 years, I have been freelancing: selling my skills as a free agent. I do very little freelancing these days, but I still regularly give advice to students and colleagues. Is it wise? I let you be the judge: here are my favorite bits of wisdom.

There are bad and good clients.

In retail, the more clients you have, the better. The economics of freelancing are different. That is because you are selling finite ressources (your time and your energy) and every transaction depletes your ressources.

As a freelancer, it is entirely reasonable to turn down work. It is often the strategic thing to do, even if you have nothing else lined up. Think about an actor offered the leading role in a major movie that is bound to be a failure.

Ultimately, the great thing about being a freelancer is the freedom to turn down work. It is not only good business, but it is also what sets you apart from employees.

Everything is negotiable.

When I started freelancing, some clients would put forward rules or policies. These rules were invariably convenient to my clients.

For example, it is common to have bounds on how much consultants can charge. A few times over the years, even recently, a client told me that I could not charge over $50 an hour, as a rule. Whatever the rule or the policy, it is always a matter of negotiation. Do not worry, clients will make “exceptions” if you are worth it. Intellectual property is another important point… when freelancing, you should not sign away your rights lightly. For example, if you are doing a programming job, consider that giving the client the copyright of your work might prevent you from reusing the same code in other projects. A much more reasonable default stance is to license your work to the client. In all cases, remember that there are bad and good clients. If a client refuses to negotiate in good faith, he may not be a good client to you. Do not watch your clock. Because it is a widespread practice, clients almost always want you to charge by the hour. Often, they want to know ahead of time how many hours you will charge. Charging by the hour is more of a metaphore. In practice, you should charge by the value provided. That is, suppose that you can solve a problem in 5 minutes but that 99.9% of world experts would take 5 days… then it makes no sense to charge 5 minutes. Similarly, if a client wants you to do some work that most people can do in a few minutes but that will take you days, you should not charge the client a lot. A more reasonable approach is to charge flat fees or the equivalent. A flat fee could include a service such as “being available at a moment’s notice”. If it reassures the client, you can translate the flat fee into a fixed number of hours and an hourly rate. Whatever you are charging, you should not worry about the time you spend on projects too much. Your main worry should be to provide something valuable to your clients. StackOverflow How to use Node.js cluster with ZeroMQ reply workers The basic question is "how to setup a ZeroMQ REPLY socket as a worker?". In effect, how to swap out the default example using an HTTP server with a ZeroMQ REPLY server, eg.: var cluster = require('cluster'), zmq = require('zmq'); if (cluster.isMaster) { cluster.fork(); cluster.fork(); } else { // Using http.createServer(..).listen(5555); works perfectly // However, the following does not: var socket = zmq.socket('rep'); socket.bind( "tcp://*:5555", function (err) { console.log(process.pid, err); } ); }  The first worker logs 'undefined' (ie. no error), whereas the second worker logs an error: Address already in use. From the "How it works" docs, the bit that seems appropriate is here (emphasis added): When you call server.listen(...) in a worker, it serializes the arguments and passes the request to the master process. If the master process already has a listening server matching the worker's requirements, then it passes the handle to the worker. If it does not already have a listening server matching that requirement, then it will create one, and pass the handle to the worker. How to "match requirements" with a ZeroMQ REPLY socket? Create a scala function to define Gatling HTTP request chains I'm just a newbie in Scala and gatling. When I define Object A{ val asset_sort = exec(http("Sort by Displays") .get("/data/analytics/reports/") .queryParamMap(asset_details_map) .queryParam("""sort""", """video_starts""") .check(status.is(200))) .pause(1) val device_sort = and so on ... variables. }  Now I want to create a Scala function which return me different variable by passing certain params. I tried something like this val asset_sort = generateWebRequest("Sort by Displays", "video_starts", asset_details_map) def generateWebRequest(requestName:String, sort:String, queryParamMap:HashMap):ChainBuilder = { return exec(http(requestName) .get("/data/analytics/reports/") .queryParamMap(queryParamMap) .queryParam("""sort""", sort) .check(status.is(200))) .pause(1) }  But it throws error i_ui\lib\AssetDetail.scala:47: class HashMap takes type parameters 12:50:36.708 [ERROR] i.g.a.ZincCompiler$ -   def generateWebRequest(requestName:String, sort:String, qu
eryParamMap:HashMap):ChainBuilder = {
12:50:36.710 [ERROR] i.g.a.ZincCompiler$-  TheoryOverflow Hardness of UNAMBIGUOUS-3DM Let UNAMBIGUOUS-3DM be defined by analogy to UNAMBIGUOUS-SAT, i.e. as a promise problem version of three-dimensional matching where we may assume there is no more than one solution. 1. Is there a randomised reduction from SAT, as there is for UNAMBIGUOUS-SAT, so that a polynomial-time algorithm for UNAMBIGUOUS-3DM would imply NP=RP? For example, it would suffice for there to be a parsimonious reduction from SAT to 3DM, but I haven't found one in the literature. 2. If so, does it still work when restricting to planar instances of 3DM, i.e. where the bipartite graph associated with the 3DM instance is planar? StackOverflow installing JDK8 on Windows XP - advapi32.dll error I downloaded JDK8 build b121 and while trying to install I'm getting the following error: the procedure entry point RegDeleteKeyExA could not be located in the dynamic link library ADVAPI32.dll The operating system is Windows XP, Version 2002 Service Pack 3, 32-bit. CompsciOverflow What are the axioms, inference rules, and (formal) semantics of a lamba calculus? Wikipedia says that a lambda calculus is a formal system. It defines a lambda calculus by giving its alphabet, and inductively describing what is inside its formal language. 1. Since a lambda calculus is a formal system, what are its • axioms and • inference rules? 2. Does a lambda calculus have semantics? If yes, how does an interpretation of a lambda calculus look like as a mapping from what subset to another? /r/scala Scala Times - a free, once-weekly, non-marketing Scala newsletter Planet Theory “How Might Quantum Information Transform Our Future?” So, the Templeton Foundation invited me to write a 1500-word essay on the above question. It’s like a blog post, except they pay me to do it! My essay is now live, here. I hope you enjoy my attempt at techno-futurist prose. You can comment on the essay either here or over at Templeton’s site. Thanks very much to Ansley Roan for commissioning the piece. /r/emacs master emacs in one year StackOverflow Play json parse missing field as empty array Using play-json 2.3 How do i parse this {i: 0}  into this case class A(i: Int, s: Seq[Int])  i would very much like to reuse Json.format macro at the moment it gives me "missing path s" UPD: i ended up writing a custom format and pimping it onto JsPath: implicit class JsPathExtensions(path: JsPath) { //return mzero if the path is missing def lazyFormatNullableM[T: Monoid](f: => Format[T]): OFormat[T] = OFormat(path.lazyReadNullable(f).map(_.orZero), new OWrites[T] { override def writes(o: T) = path.lazyWriteNullable(f).writes(o.some) }) }  UnixOverflow How much RAM needed for preventing use swap? At my system always available some "Free" and "Inactive" memory, but sometimes system move couple megabytes to swap. Per night it maybe around 100MB, but for a few days in swap could be more than 4000MB. How can I determine required memory for my system? StackOverflow Scala type parameter inference fails for Nothing scala> class A[T] defined class A scala> class B[T](a: A[T]) defined class B scala> val b = new B(new A[Int]) b: B[Int] = B@1ff8982d  Great! I can create instances of B from instances of A. With one exception. For instances of A[Nothing] the type inference fails. scala> val b = new B(new A[Nothing]) <console>:9: error: type mismatch; found : A[Nothing] required: A[T] Note: Nothing <: T, but class A is invariant in type T. You may wish to define T as +T instead. (SLS 4.5) val b = new B(new A[Nothing])  Specifying the type manually works. scala> val b = new B[Nothing](new A[Nothing]) b: B[Nothing] = B@3aad5958  Should I file a bug for it, or is it an intentional trap to scare away programmers that lack sufficient resolve for Scala? update the last element of List I have a List val first = List("A","B","C","D")  and I want to create a new list from it but change the last element only: val newLastVal = "E" val second = List("A","B","C","E")  can't figure this one out! Thanks in advance Indexing Scala Case Classes in GridGain I am trying to use Scala Case Classes as the object in Gridgain caches. But I cant get it to work. It works perfectly fine if I create exactly the same object as a java-class. This is my testcase: class GridGainTest extends FunSuite { val grid = GridGain.start("gridgain_mini.xml") private val theCache = grid.cache[Long, TestData2]("theCache") test("That the basics is working") { val tx = theCache.txStart() theCache.put(1, new TestData2(1, "Hello", 0)) theCache.put(2, new TestData2(2, "World", 1)) tx.commit() val q = theCache.queries().createSqlQuery(classOf[TestData2], "parent = ?") val qRes1 = q.execute(new java.lang.Long(1)).get() assert(qRes1.size() == 1) } } @BeanInfo case class TestData(@BeanProperty @GridCacheQuerySqlField(index = true) id: Long, @BeanProperty @GridCacheQuerySqlField(index = true) name: String, @BeanProperty @GridCacheQuerySqlField(index = true) parent: Long) public class TestData2 { @GridCacheQuerySqlField(index = true) public final long id; @GridCacheQuerySqlField(index = true) public final String name; @GridCacheQuerySqlField(index = true) public final long parent; public TestData2(long id, String name, long parent) { this.id = id; this.name = name; this.parent = parent; } }  With TestData2 the test passes, but with TestData it fails on the assert due to size = 0. I have tried several combinations of property-types (java.lang.Long etc) and annotation-combinations in the case class but nothing seems to work. I must miss something fundamental here, but a few hours of tests from my side obviously doesnt get me all the way. Any hints? cond in Clojure with thousands of clauses Running the following code in Clojure gives a StackOverflow Error: (cond (= 1 2) 1 (= 2 3) 2 (= 3 4) 3 ... (= 1022 1023) 1022 (= 1023 1024) 1023 :else 1024)  I would like to create a function/macro that can handle a huge number of clauses without creating a stack that overflows. Please advise as to how I might attempt this. Lobsters Stop struggling with Python on Windows CompsciOverflow What are practical applications of AI Planning? [on hold] note: any tips toward making this more constructive will be highly appreciated. I've dealt with ai-related problems, from searching algorithms, to computer vision, to machine learning. However none of which required planning, just algorithms following algorithms. I'm not sure how effective will it be to implement AI Planning in practical problems, so I'm curious to know if there's a significant effect in doing so. Like at what point does the programmer/designer realize we need to apply AI Planning to solve this. /r/emacs Manually Choose a Fallback Font for Unicode StackOverflow Scala :- Gatling :- Concatenation of two Maps stores last value only and ignores all other values I have a two Maps and I want to concatenate them. I tried almost all example given here Best way to merge two maps and sum the values of same key? but it ignores all values for key metrics and only stores last value. I have downloaded scalaz-full_2.9.1-6.0.3.jar and imported it import scalaz._ but it won't works for me. How can I concate this two maps with multiple values to same keys ? Edit :- Now I tried  val map = new HashMap[String, Set[String]] with MultiMap[String, String] map.addBinding("""report_type""" , """performance""") map.addBinding("""start_date""" ,start_date) map.addBinding("""end_date""" , end_date) map.addBinding("metrics" , "plays") map.addBinding("metrics", "displays") map.addBinding("metrics" , "video_starts") map.addBinding("metrics" , "playthrough_25") map.addBinding("metrics", "playthrough_50") map.addBinding("metrics", "playthrough_75") map.addBinding("metrics", "playthrough_100") val map1 = new HashMap[String, Set[String]] with MultiMap[String, String] map1.addBinding("""dimensions""" , """asset""") map1.addBinding("""limit""" , """50""")  And tried to conver this mutable maps to immutable type using this link as val asset_query_string = map ++ map1 val asset_query_string_map =(asset_query_string map { x=> (x._1,x._2.toSet) }).toMap[String, Set[String]]  But still I get i_ui\config\config.scala:51: Cannot prove that (String, scala.collection.immutable.Set[String]) <:< (St ring, scala.collection.mutable.Set[String]). 11:10:13.080 [ERROR] i.g.a.ZincCompiler$ - val asset_query_string_map =(asset_query_string map { x=> (x
._1,x._2.toSet) }).toMap[String, Set[String]]


Planet Emacsen

Irreal: Fast Toggling Emacs Modes

Over at Endless Parentheses, which I’ve mentioned before, Artur Malabarba has a really good idea. He shows how to define a custom key map that he uses to toggle modes that he uses frequently. He binds the commands to the prefix【Ctrl+x t】, an unused key sequence in stock Emacs. Then he has a third key that mnemonically suggest the mode he wants to toggle. Thus,【Ctrl+x t c】toggles column-number-mode and【Ctrl+x t r】toggles read-only-mode and so on.

Oddly, none of the shortcuts that he proposes are ones I care about but I do have a list of my own

 l linum-mode o org-mode p paredit-mode t text-mode v visual-line-mode

As Malabarba says, these aren’t keys I’ll use everyday but they are mnemonic enough that I’ll be able to remember then.

Doubtless you have your own list of modes you’d like to be able to toggle easily and Malabarba’s method makes it easy to bind them to an easily remembered key.

A very useful post, I think. I’m going to implement the above list in my own init.el as soon as I finish writing this.

Background

I am trying to provision a multi-tier distributed application on OpenStack private cloud using the Ansible provided modules for Nova and Quantum. This application has a number of moving parts:

• a Hadoop cluster
• a MongoDB cluster
• a web application that makes use of the above

Based on these tiers, I want to:

• provision a specified number of hosts in several logical groups relating to each tier
• use different instance flavors based on characteristics of the associated tier
• create each group in parallel to minimize duration

One thing to note is that the number of instances should be configurable at provisioning time, much the same way that is done for AWS EMR.

Question

Is there a way to perform the following sketch of actions in Ansible assuming a tier group of 5:

< start all 5 at once >
< start up openstack host >
< assign it a floating ip >
< capture the floating ip >
< end when all 5 have a floating ip >
< wait for all 5 floating ips to have an open SSH port >
< use all floating ips within a group to issue more actions >


The final step is important as it allows for a join point to apply further processing to a named group.

References

There is a thread that discusses parallelism in some detail, but fails to offer any resolution of the final step in the sketch outlined above.

For the benefit of the reader, I have compiled a small set of related examples that are similar to what I am trying to achieve in that they make use of the Nova and Quantum module actions (which alone are quite scarce). However, they do not seem to leverage parallelism or coordinate multiple groups of hosts:

Planet Theory

TR14-091 | One time-travelling bit is as good as logarithmically many | Ryan O&#39;Donnell, A. C. Cem Say

We consider computation in the presence of closed timelike curves (CTCs), as proposed by Deutsch. We focus on the case in which the CTCs carry classical bits (as opposed to qubits). Previously, Aaronson and Watrous showed that computation with polynomially many CTC bits is equivalent in power to PSPACE. On the other hand, Say and Yakary?lmaz showed that computation with just 1 classical CTC bit gives the power of “postselection”, thereby upgrading classical randomized computation (BPP) to the complexity class BPP path and standard quantum computation (BQP) to the complexity class PP. It is natural to ask whether increasing the number of CTC bits from 1 to 2 (or 3, 4, etc.) leads to increased computational power. We show that the answer is no: randomized computation with logarithmically many CTC bits (i.e., polynomially many CTC states) is equivalent to BPP_path. (Similarly, quantum computation augmented with logarithmically many classical CTC bits is equivalent to PP.) Spoilsports with no interest in time travel may view our results as concerning the robustness of the class BPP_path and the computational complexity of sampling from an implicitly defined Markov chain.

StackOverflow

When should I use function currying in Python?

When should I write my functions in curried form? does not match my thought, need to correct myself.

As part of my learning link, this is what I understand from function currying. Below is one example:

def curry2(f):
"""Returns a function g such that g(x)(y) == f(x, y)

>>> from operator import add
"""
def g(x):
def h(y):
return f(x, y)
return h
return g


In any application, if I know that the number of arguments are fixed (say 2 arguments) and function name is normalise_range(say), then I will define def normalise_range(x, y): function and use it in my application directly by calling normalise_range(x, y).

In any application, if I know that, the number of arguments are fixed (say 2 arguments), but the function name is varying (can be normalise_range/average/I don't know..), then I will use def curry2(f): as shown above, which will accept all functions that take two arguments (fixed).

My question:

1. Is my understanding correct?
2. If yes, can we think of currying for functions of variable number of arguments?

CompsciOverflow

Finding a node that is minimal w.r.t. maximum distance to any other node [on hold]

Given a graph and an edge in the graph, I want to find a point on this edge that makes the maximum distance from all the nodes to this point minimized. Is there any graph-theory based algorithm that can solve such problem rather than brute force?

For example we have a graph:

1 2 10
2 3 10
3 4 1
4 1 5


In every line the first two values indicate the node number, and the 3rd one is the corresponding edge value. Now I want to find a point on the edge 1-2, the answer is a point with distance of 2 to node 1 (8 to node 2). And 1-p distance is 2, 2-p is 8, 3-p is 1+5+2=8, 4-p is 5+2=7. The maximum path is 8, which is the minimal among all the possible points p on edge 1-2.

StackOverflow

How to avoid adding a 'root' scala sbt project in IntelliJ when defining github dependencies?

I'm using IntelliJ with SBT plugin and I've added the following lines to build.sbt in order to add a dependency to a private github repository:

lazy val g = RootProject(uri("ssh://git@github.com/XXXX/myrepo.git"))

lazy val root = project in file("myproject") dependsOn g


After running sbt, my referenced project is successfully cloned, but build fails due to output path clashes between my base project and a root project that is automatically added each time I refresh sbt after modifying build.sbt.

What is coming up for scala in 2.10? [closed]

Obviously there is the reflection library but what else can we expect to see in the next release of scala? Any language changes? Is there a website where I can see a roadmap?

CompsciOverflow

If Halting problem is decidable, are all RE languages decidable? [on hold]

Assume the halting problem was decidable. Is then every recursively enumerable languagerecursive?

Background:

Decision tree complexity or query complexity is a simple model of computation defined as follows. Let $f:\{0,1\}^n\to \{0,1\}$ be a Boolean function. The deterministic query complexity of $f$, denoted $D(f)$, is the minimum number of bits of the input $x\in\{0,1\}^n$ that need to be read (in the worse case) by a deterministic algorithm that computes $f(x)$. Note that the measure of complexity is the number of bits of the input that are read; all other computation is free.

We define the bounded error or Las Vegas randomized query complexity of $f$, denoted $R_0(f)$, as the minimum number of input bits that need to be read in expectation by a zero-error randomized algorithm that computes $f(x)$. A zero-error algorithm always outputs the correct answer, but the number of input bits read by it depends on the internal randomness of the algorithm. (This is why we measure the expected number of input bits read.)

We define the bounded error or Monte Carlo randomized query complexity of $f$, denoted $R_2(f)$, to be the minimum number of input bits that need to be read by a bounded-error randomized algorithm that computes $f(x)$. A bounded-error algorithm always outputs an answer at the end, but it only needs to be correct with probability $\geq$ $1 - \delta$ ($2/3$, say).

Work on Recursive Boolean Functions:

There has been a line of work on the decision tree complexity of recursive boolean functions as mentioned below. The techniques focus on applying Yao's Lemma and using the distributional perspective guaranteed by it. This means we define a probability distribution on the inputs and the cost incurred by the best algorithm for this distribution gives a lower bound on the randomized decision tree complexity of the function. The worst possible distribution will give the actual randomized decision tree complexity.

The techniques in these works focus on giving a lower bound on the cost incurred by reading the "minority" bits (or vertices in the function tree) of the input via some form of induction. Another direction of attack could be to find the most "hard" distribution.

Some Notions

We define: The distribution $D^*$ on an input set $I$ is hard for a given function $f$, if $\forall D$ on $I$, $C(A,D) \leq C(A^*, D^*)$, where $C(A,D)$ is the expected cost (i.e. number of input bits read on expectation) incurred by the deterministic decision tree $A$ when the input follows the probability distribution is $D$. where $A^* = \operatorname{argmin}_A C(A, D^*), A = \operatorname{argmin}_A C(A, D)$

A distribution $D_1 < D_2$, if $C_m(D_1) < C_m(D_2)$, where $C_m(D_i) = C(A_i, D_i)$, and $A_i = \operatorname{argmin}_A C(A, D_i)$ . In other words $D_2$ is harder than $D_1$ means the best possible algorithm for $D_2$ does worse than the best possible algorithm for $D_1$. Note: The algorithm must be correct in the whole domain, and not just in the support of the distributions. For the base case of a recursive boolean function like say 2 bits or 4 bits, it is often easy to show a certain distribution to be hard. Often it is an easy observation or an obvious fact. In many cases, it may seem natural that the "hard" distribution is the recursive extension of the hard distribution. However, this may not be true in general, especially if the function is not symmetric over the input bits and rather skewed i.e. not all input bits are equally important to infer the value of the function on certain inputs (or a subset thereof).

Questions:

Is there any work on how to approach the problem of finding the "hard" distribution of the recursive function, from that of the base case function?

Is there any interesting connection of this problem with any other problems? Any comments are welcome.

References:

[1] M. Saks, A. Wigderson Probabilistic Boolean Decision Trees and the Complexity of Evaluating Game Trees Proceedings of the 27th Foundations of Computer Science, pp. 29-38, October 1986.

[2] Frédéric Magniez, Ashwin Nayak, Miklos Santha, and David Xiao. Improved bounds for the randomized decision tree complexity of recursive majority. In Luca Aceto, Monika Henzinger, and Jiri Sgall, editors, ICALP (1), volume 6755 of Lecture Notes in Computer Science, pages 317–329. Springer, 2011.

[4] Nikos Leonardos. An improved lower bound for the randomized decision tree complexity of recursive majority. In Fedor V. Fomin, Rusins Freivalds, Marta Z. Kwiatkowska, and David Peleg, editors, Proceedings of 40th International Colloquium on Automata, Languages and Programming, volume 7965 of Lecture Notes in Computer Science, pages 696{708. Springer, 2013.

StackOverflow

Play Framework: Netty fails to bind after recompile

I am working on a project in the Play Framework using Akka and Memcached . When I change the java source in the IDE, play automatically re compiles and reruns/reloads. There is code in our project that looks for a free port for Akka. It starts at 40000. Tries to open and close a socket on the port. On success, it constructs a configuration for Akka including found available ports. But later this initialization fails throwing this exception:

 Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /127.0.0.1:40000

at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at akka.remote.transport.netty.NettyTransport$$anonfunlisten1.apply(NettyTransport.scala:377) at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:374)
at scala.util.Success$$anonfunmap1.apply(Try.scala:206) at scala.util.Try.apply(Try.scala:161) at scala.util.Success.map(Try.scala:206) at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
at scala.concurrent.Future$$anonfunmap1.apply(Future.scala:235) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at akka.dispatch.BatchingExecutorBatch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) at akka.dispatch.BatchingExecutor$Batch$$anonfunrun1.applymcVsp(BatchingExecutor.scala:82) at akka.dispatch.BatchingExecutorBatch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at akka.dispatch.BatchingExecutor$Batch$$anonfunrun1.apply(BatchingExecutor.scala:59) at scala.concurrent.BlockContext.withBlockContext(BlockContext.scala:72) at akka.dispatch.BatchingExecutorBatch.run(BatchingExecutor.scala:58) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) at akka.dispatch.ForkJoinExecutorConfiguratorAkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPoolWorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:344) at sun.nio.ch.Net.bind(Net.java:336) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:199) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.jboss.netty.channel.socket.nio.NioServerBossRegisterTask.run(NioServerBoss.java:193) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290) at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutorWorker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722)  Strange thing is that this code works on Ubuntu, but does not on Mac OSX (Maverics). Any idea would be most appreciated. Fefe Habt ihr das auch gehört, wie würdelos die fiesen ... Habt ihr das auch gehört, wie würdelos die fiesen Rebellen mit den Leichen der Flugzeugabschussopfer umgegangen sind? Fies und Würdelos! Das hat Kerry aus Washington diagnostiziert und die Weltpresse hat es unhinterfragt berichtet. Auch der ORF hat dann mal ihren Korrespondenten vor Ort angerufen, und wollte wissen, ob die fiesen Rebellen endlich aufgehört haben, die Leichen so würdelos zu behandeln. Das Interview lief dann nicht ganz so, wie sich die ORF-Redakteure sich das vorgestellt hatten. StackOverflow Scala Ordered Mixin Base Class In Scala 2.10.2, I have an abstract base class for a major-minor style version number: abstract class MajorMinorVersion( private [version] val major : Int, private [version] val minor : Int ) extends Ordered[MajorMinorVersion] { override def toString() : String = "%d.%d".format( major, minor ) def compare( that : MajorMinorVersion ) = that.major == major match { case false => major - that.major case true => minor - that.minor } }  Several custom file types I'm reading have a version number in this format: case class FooFileVersion( maj : Int, min : Int ) extends MajorMinorVersion( maj, min ) case class BarFileVersion( maj : Int, min : Int ) extends MajorMinorVersion( maj, min )  This works nicely in that: scala> FooFileVersion( 1, 3 ) < FooFileVersion( 1, 4 ) res0: Boolean = true  The problem with this approach is that I'd like comparison of FooFileVersion and BarFileVersion instances to throw an error, because comparing them doesn't make sense, but currently we have: scala> FooFileVersion( 1, 3 ) < BarFileVersion( 1, 4 ) res0: Boolean = true  What's the best way to get around this? Is it possible to have the Ordered trait mixed in to the base class, or will I need to mix it in to the inheriting classes, and define the compare method there? That would be a solution, I guess, but it would be nice to be able to just abstract the comparison away into a base class. I'm sure there's a relatively simple solution somewhere I'm just missing. Thanks in advance! CompsciOverflow Limit on no. of emails I can RECEIVE on my free gmail account I would like to know what's the limit on number of emails I can receive on my Gmail﻿ account( a free gmail account that ends with @gmail.com) in a day! Thanks in advance. Please provide help ASAP. What is the signal going from the computer to the screen? Im sorry if it is a weird question, but I cant seem to find an answer for this. What does the signal going to the screen look like in more abstract sense? Does it code the values of each pixel in a huge matrix, is it a computational stream that instructs the screen what to present? In other words, can I grab this signal by a "listening" device for example and deconstruct it with algorithms? Not just grab the signal and present it on another screen. For example, if it is a matrix of pixel values, then I can run face recognition algorithms for example, to see if the signal sent to the computer is a picture of face or not. It might also depend on the type of screen? Say that it is a LCD screen. edit: To elaborate, if I wanted to take the signal sent to the screen and scan it for text (NSA style), could I? If so, what should I know of the signal that Im collecting to actually do this? Disclaimer: Im not interested in spying on anyone. StackOverflow Trying to get a lazy-contains? function in Clojure to return results consistant with contains? when dealing with a map Here's a use of the standard 'contains?' function in Clojure- (contains? {:state "active", :course_n "law", :course_i "C0"} :state)  and it returns the expected true  I used the following Clojure: Idiomatic way to call contains? on a lazy sequence as a guide for building a lazy-contains? as this is what I need for my present use-case. The problem I'm facing is that for a map these alternatives are not returning the same answer, giving either a false or a nil response. I've tried looking at the source for contains? and it's slow going trying to understand what's happening so I can correct the lazy-contains? appropriately (for the record Clojure is essentially my first programming language, and my exposure to Java is very limited). Any thoughts or ideas on how I might approach this? I tried every variant on the linked question I could. Thanks in advance. CompsciOverflow Does a NFA accept context free languages? [on hold] A NFA accepts regular languages (Type-3). So because of the chomsky hierarchy every regular language is also context-free. So a NFA accepts context free languages. Am I wrong? Moreover a NFA accepts through definition every language. TCP & UDP time taken/retransmission calculation What i know is that, TCP provides reliable point-to point communication, whereas UDP doesn't establish a connection before sending data, it just sends. For this question, assuming that for UDP-IPv4 communication the overhead is 66bytes, and TCP-IPv4 overhead is 78 bytes. And for the calculation in IPv6, i came out with two option. Option 1: Version- 4bits Traffic Class- 8bits Flow label- 20 bits Payload Length- 16 bits Next header- 8 bits Hop Limit- 8 bits Source Address- 128 bits Destination Address- 128 bits Total = 320 bits = 40 bytes TCP over IPv6: 20 bytes (TCP) + 40 bytes (IPv6) = 60 bytes Option 2: By using the overhead of IPv4 (78 bytes) + TCP (20 bytes) = 98 Bytes The reason why i have option 2 is because IPv6 is 20 bytes larger than IPv4. So i'm not sure which option is valid. Can anyone enlighten me? StackOverflow How to Embed a jLine element within TextArea in Scala Swing I want to embed the interactive shell within a TextArea element of swing application. Why I can't do the following: object embedInteraction extends SimpleSwingApplication { def top = new MainFrame { contents = gui } val gui = new BoxPanel(Orientation.Vertical) { //Other GUI Elements Terminal } def Terminal { val comGetter = new jline.console.ConsoleReader() while(!comGetter == "q"){ //Deals with other commands here } } }  Fefe Die Ausweichflugroute von Malaysia Airlines geht über ... RT: Wie aus dem trauernden Rebellen ein fieser Plünderer ... /r/emacs Problem with the Python REPL in emacs Let's say I have the following python code: file a.py from b import test def test_a(): test()  file b.py def test(): print "b"  Now I open a.py and press C-c C-c. It opens the python REPL and I can run test_a() inside. If I go to the file b.py, replace print "b" by print "bb", and press C-c C-c, I can run test_a() in the REPL and I get "bb". For now it's all fine. Now if I go back to a.py, and press C-c C-c, I get "b" while running test_a() in the REPL. The REPL gets the old version of the file b.y. Has someone already experimented that problem? submitted by eskatrem [link] [6 comments] Fefe Erdogan lässt dann mal über 50 hochrangige Polizisten ... TheoryOverflow Applications of Harrow's algorithm for solving linear equations In Harrow's algorithm for solving a system of linear equations the output is a quantum state rather than explicit information. Has anyone been able to apply knowledge of this quantum state to solve a problems which would classically use the solutions of the linear equations? DataTau Arithmetic on data.frame objects in R StackOverflow What's the essential similarities and differences between Scala and Gosu (related to Java)? I am studying Scala right now, and have been doing so for the last month. Today, (via Twitter) I stumbled upon another language that seems to be quite similar, Gosu. What are the essential similarities and differences between Scala and Gosu with respect to their most influential parent, Java? Are there any web-sites that compare and contrast these two languages? reading cookie value in play framewrok 2.2 scala template I am trying to read a cookie value inside the play framework template (not inside a controller). I am trying the following which is not working: @ val cookieVal = request.cookies.get('PLAY_SESSION').value  Any suggestions to fix this will be greatly appreciated. The reason why I am trying this is to change how the page gets rendered based on a cookie value. Spark Streaming Stateful Network Word Count This is the example code that came with Spark. I copied the code here and this is the link to it: https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/StatefulNetworkWordCount.scala. However, when I was trying to run the program using command "bin/run-example org.apache.spark.examples.streaming.StatefulNetworkWordCount localhost 9999", I was given the following error: 14/07/20 11:52:57 ERROR ActorSystemImpl: Uncaught fatal error from thread [spark-akka.actor.default-dispatcher-4] shutting down ActorSystem [spark] java.lang.NoSuchMethodError: java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMapKeySetView; at org.apache.spark.streaming.scheduler.JobScheduler.getPendingTimes(JobScheduler.scala:114) at org.apache.spark.streaming.Checkpoint.(Checkpoint.scala:43) at org.apache.spark.streaming.scheduler.JobGenerator.doCheckpoint(JobGenerator.scala:259) at org.apache.spark.streaming.scheduler.JobGenerator.orgapachesparkstreamingschedulerJobGenerator$$processEvent(JobGenerator.scala:167) at org.apache.spark.streaming.scheduler.JobGenerator$$anonfunstart1$$anon$1$$anonfunreceive1.applyOrElse(JobGenerator.scala:76) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) at akka.actor.ActorCell.invoke(ActorCell.scala:456) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) at akka.dispatch.Mailbox.run(Mailbox.scala:219) at akka.dispatch.ForkJoinExecutorConfiguratorAkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPoolWorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Exception in thread "Thread-37" org.apache.spark.SparkException: Job cancelled because SparkContext was shut down at org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply(DAGScheduler.scala:639) at org.apache.spark.scheduler.DAGScheduler$$anonfuncleanUpAfterSchedulerStop1.apply(DAGScheduler.scala:638) at scala.collection.mutable.HashSet.foreach(HashSet.scala:79) at org.apache.spark.scheduler.DAGScheduler.cleanUpAfterSchedulerStop(DAGScheduler.scala:638) at org.apache.spark.scheduler.DAGSchedulerEventProcessActor.postStop(DAGScheduler.scala:1215) at akka.actor.dungeon.FaultHandlingclass.akkaactordungeonFaultHandling$$finishTerminate(FaultHandling.scala:201) at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:163) at akka.actor.ActorCell.terminate(ActorCell.scala:338) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:431) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:447) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:262) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:240) at akka.dispatch.Mailbox.run(Mailbox.scala:219) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) 14/07/20 11:53:00 ERROR Executor: Exception in task ID 0 java.lang.IllegalStateException: cannot create children while terminating or terminated at akka.actor.dungeon.Children$class.makeChild(Children.scala:184) at akka.actor.dungeon.Children$class.attachChild(Children.scala:42) at akka.actor.ActorCell.attachChild(ActorCell.scala:338) at akka.actor.ActorSystemImpl.actorOf(ActorSystem.scala:518) at org.apache.spark.streaming.receiver.ReceiverSupervisorImpl.(ReceiverSupervisorImpl.scala:67) at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun9.apply(ReceiverTracker.scala:263) at org.apache.spark.streaming.scheduler.ReceiverTrackerReceiverLauncher$$anonfun$9.apply(ReceiverTracker.scala:257) at org.apache.spark.SparkContext$$anonfunrunJob4.apply(SparkContext.scala:1080) at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1080) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111) at org.apache.spark.scheduler.Task.run(Task.scala:51) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:187) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 14/07/20 11:53:06 ERROR ExecutorUncaughtExceptionHandler: Uncaught exception in thread Thread[spark-akka.actor.default-dispatcher-13,5,main] org.apache.spark.SparkException: Error sending message to BlockManagerMaster [message = HeartBeat(BlockManagerId(, x-131-212-225-148.uofm-secure.wireless.umn.edu, 47668, 0))] at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:251) at org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:51) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:113) at org.apache.spark.storage.BlockManager$$anonfun$initialize$1$$anonfunapplymcVsp1.applymcVsp(BlockManager.scala:158) at org.apache.spark.util.Utils.tryOrExit(Utils.scala:790) at org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:158) at akka.actor.Scheduler$$anon9.run(Scheduler.scala:80) at akka.actor.LightArrayRevolverScheduler$$anon$3$$anon2.run(Scheduler.scala:241) at akka.actor.LightArrayRevolverSchedulerTaskHolder.run(Scheduler.scala:464) at akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281) at akka.actor.LightArrayRevolverScheduler$$anonfunclose1.apply(Scheduler.scala:280) at scala.collection.Iteratorclass.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) at scala.collection.IterableLikeclass.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279) at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anonfun_start1.apply(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anon3.run(ActorSystem.scala:596) at akka.actor.ActorSystemImplTerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfunrun1.applymcVsp(ActorSystem.scala:753) at akka.actor.ActorSystemImplTerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfunrun1.apply(ActorSystem.scala:746) at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15) at akka.actor.ActorSystemImplTerminationCallbacks.run(ActorSystem.scala:746) at akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) at akka.actor.ActorSystemImpl$$anonfunterminationCallbacks1.apply(ActorSystem.scala:593) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at akka.dispatch.BatchingExecutorBatch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) at akka.dispatch.BatchingExecutor$Batch$$anonfunrun1.applymcVsp(BatchingExecutor.scala:82) at akka.dispatch.BatchingExecutorBatch$$anonfun$run$1.apply(BatchingExecutor.scala:59) at akka.dispatch.BatchingExecutor$Batch$$anonfunrun1.apply(BatchingExecutor.scala:59) at scala.concurrent.BlockContext.withBlockContext(BlockContext.scala:72) at akka.dispatch.BatchingExecutorBatch.run(BatchingExecutor.scala:58) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) at akka.dispatch.ForkJoinExecutorConfiguratorAkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPoolWorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: akka.pattern.AskTimeoutException: Recipient[Actor[akka://spark/user/BlockManagerMaster#1887396223]] had already been terminated. at akka.pattern.AskableActorRef.askextension(AskSupport.scala:134) at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:236) **************CODE****************** object StatefulNetworkWordCount { def main(args: Array[String]) { if (args.length < 2) { System.err.println("Usage: StatefulNetworkWordCount <hostname> <port>") System.exit(1) } StreamingExamples.setStreamingLogLevels() val updateFunc = (values: Seq[Int], state: Option[Int]) => { val currentCount = values.foldLeft(0)(_ + _) val previousCount = state.getOrElse(0) Some(currentCount + previousCount) } val sparkConf = new SparkConf().setAppName("StatefulNetworkWordCount") // Create the context with a 1 second batch size val ssc = new StreamingContext(sparkConf, Seconds(1)) ssc.checkpoint(".") // Create a NetworkInputDStream on target ip:port and count the // words in input stream of \n delimited test (eg. generated by 'nc') val lines = ssc.socketTextStream(args(0), args(1).toInt) val words = lines.flatMap(_.split(" ")) val wordDstream = words.map(x => (x, 1)) // Update the cumulative count using updateStateByKey // This will give a Dstream made of state (which is the cumulative count of the words) val stateDstream = wordDstream.updateStateByKey[Int](updateFunc) stateDstream.print() ssc.start() ssc.awaitTermination() } }  I wonder if it is because, it is trying to set up the checkpoint at my local file system by doing commands "ssc.checkpoint(".")", while the file is not a file compatible with hadoop? (the file must be compatible with hadoop in order to set the checkpoint) If it is, how could I fix it? Thanks! CompsciOverflow TCP & UDP time taken/retransmission calculation What i know is that, TCP provides reliable point-to point communication, whereas UDP doesn't establish a connection before sending data, it just sends. For this question, assuming that for UDP-IPv4 communication the overhead is 66bytes, and TCP-IPv4 overhead is 78 bytes. And for the calculation in IPv6, i came out with two option. Option 1: Version- 4bits Traffic Class- 8bits Flow label- 20 bits Payload Length- 16 bits Next header- 8 bits Hop Limit- 8 bits Source Address- 128 bits Destination Address- 128 bits Total = 320 bits = 40 bytes TCP over IPv6: 20 bytes (TCP) + 40 bytes (IPv6) = 60 bytes Option 2: By using the overhead of IPv4 (78 bytes) + TCP (20 bytes) = 98 Bytes The reason why i have option 2 is because IPv6 is 20 bytes larger than IPv4. So i'm not sure which option is valid. Can anyone enlighten me? StackOverflow Designing hardware and software for Nest thermostat [on hold] For the Nest thermostat, what hardware is needed to build one of those devices (ie touchscreen)? I will try to be as specific as possible. For the components of the nest thermostat ( shown in this image : http://cdn.iphoneincanada.ca/wp-content/uploads/2013/04/nest-teardown-2.jpg ) could someone explain what each part is called and some of the functions of each part? I have updated this question to make it more specific. Receiving "bad symbolic reference to org.json4s.JsonAST..." in Eclipse Scala-IDE using 2.11 I am attempting to use the Eclipse Scala-IDE for 2.11 (downloaded the prepackaged bundle from the web-site). I have been using the Scala Worksheet to work with a SaaS API returning JSON. I've been pushing through just using String methods. I decided to begin using json4s. I went to http://mvnrepository.com/ and obtained the following libraries: • json4s-core-2.11-3.2.10.jar • json4s-native-2.11-3.2.10.jar • paranamer-2.6.jar I have added all three jars to the Project's Build Path. And they appear under the project's "Referenced Libraries". I have the following code in a Scala Worksheet: package org.public_domain import org.json4s._ import org.json4s.native.JsonMethods._ object WorkSheet6 { println("Welcome to the Scala worksheet") parse(""" { "numbers" : [1, 2, 3, 4] } """) println("Bye") }  I am receiving the following two compilation errors: • bad symbolic reference to org.json4s.JsonAST.JValue encountered in class file 'package.class'. Cannot access type JValue in value org.json4s.JsonAST. The current classpath may be missing a definition for org.json4s.JsonAST.JValue, or package.class may have been compiled against a version that's incompatible with the one found on the current classpath. • bad symbolic reference to org.json4s.JsonAST.JValue encountered in class file 'package.class'. Cannot access type JValue in value org.json4s.JsonAST. The current classpath may be missing a definition for org.json4s.JsonAST.JValue, or package.class may have been compiled against a version that's incompatible with the one found on the current classpath. When I go look in the org.json4s package in the json4s-core-2.11-3.2.10.jar file, there is in fact no .class file indicating any sort of compiled object JsonAST. This is a showstopper. Any help on this would be greatly appreciated. code completion for liftweb scala in intellij idea I'm working with a Scala application using Intellij Idea 13.1.1 community edition. Although i am able to compile and run my application, but the IDE shows error in editor window as it cannot resolve the class. How can i remove that and enable code completion feature for Scala and Liftweb? How in shapeless do you say that a proof is the empty type (i.e. false) I want to write a function that take a Nat as parameter, and return this nat ONLY if this nat isn't divisible by three. for example : def myFunction[N <: Nat](n :N)(implicit ev: /* what do I put here that say not divible by 3 ? */): N = n  To do that, I have to write something that say "N is not divisible by _3", or "Mod.Aux[N, _3,_0] is the empty type" how can I do that in shapeless ? Waiting on multiple Akka FSM messages I have an Akka FSM actor that runs the following pseudocode after receiving a message while in ReadyState lookupA ! Wrapper(Lookup("A")) lookupB ! Wrapper(Lookup("B")) lookupC ! Wrapper(Lookup("C")) goto(LookingUpDataState) using DataFound(a = None, b = None, c = None)  The actor then waits for responses which can be either FullResult[T] (extending ServiceResult[T]) or Empty (extending ServiceResult[Nothing]). Successful lookup results are used to populate the DataFound instance's fields and Empty lookup results result in a logged error message and the termination of the actor. My question is this: how can I determine which lookup failed, so that I can log the failure or fallback to a default value? All I can think of is examining the sender's ActorRef (hacky) or adding a unique ID field to all messages (high overhead). This is a simple problem to solve using Ask's and Futures. Does an idiomatic Akka solution exist? Fefe Die Rebellen haben die Black-Boxen an Malaysia übergeben. ... Die Rebellen haben die Black-Boxen an Malaysia übergeben. Die Malaysier sagen, sie seien in guter Verfassung gewesen. In einem Experiment über Lügen und Bescheißen in ... In einem Experiment über Lügen und Bescheißen in Berlin kam eine starke Korrelation zwischen Bescheißen und "im Sozialismus aufgewachsen" heraus. Die sollten würfeln und die Zahl oben oder auf der Rückseite nehmen, sich aber vor dem Würfeln festgelegt haben. Die Studie hat das aber nicht überprüft, ob die sich vorher festlegen. Und dann haben sie geguckt, ob die kleinen und großen Zahlen gleichverteilt sind. Waren sie nicht. The authors found that, on average, those who had East German roots cheated twice as much as those who had grown up in West Germany under capitalism. They also looked at how much time people had spent in East Germany before the fall of the Berlin Wall. The longer the participants had been exposed to socialism, the greater the likelihood that they would claim improbable numbers of high rolls. Wieso muss ich denn über dieses Experiment im Economist lesen und nicht in der deutschen Presse? CompsciOverflow How append, prepend, and generally insertAt work in RRB-tree I read the paper about Relaxed Radix Balanced trees (RRB trees) and am trying to implement them. What I can't get is how insertion at an index should be performed step by step. Can anyone proficient in this data structure describe this procedure? StackOverflow ClrZmq returning messages always to first started client We're creating a WPF app in which we execute python scripts from different Test Stations and show the output in its corresponding output panel, To run the scripts in parallel we are using Task but when we run the scripts in parallel from the stations, We are getting the output of other stations also into the station that is started first, we're using the following code, private void ZmqStatusListener(string endPoint) { using (Context context = new Context()) { StatusPort = string.Empty; TestResultPort = string.Empty; using (Socket server = context.Socket(SocketType.REP)) { try { if (isStatusContextActive == false || isPortChanged == true) { server.Bind(endPoint); isStatusContextActive = true; } } catch (ZMQ.Exception ex) { if (ex.Errno != 100) { string IPCPort = _globalParameters.GlbParam.GlbParamIpcStartPort; if (IPCPort == string.Empty) { IPCPort = "0"; } if (endPoint == EditorConstants.PortAddress.PortPrefix + IPCPort) { StatusPort = endPoint; TestReultError = EditorConstants.CommonMessageTypes.TestReultError + ex.Message + EditorConstants.CommonMessageTypes.StackTraceMessage + ex.StackTrace; } StopExecOfScript(default(object)); isCancelledtask = true; ScriptStatusDesc = new ScriptStatusDesc() { Status = "Failed", statusDescription = "Failed" }; } } while (true) { string message = server.Recv(Encoding.UTF8); UpdateTestResults(message); server.Send(" ACK", Encoding.UTF8); // if (message == "Test Passed") //break; } } } }  and for testing purpose we're breaking the while loop in this code based on a test message we kept in the python script, then we are able to get the output in the respective station correctly but this way we can only run in a synchronous fashion which we don't want as we require to run the test stations in parallel and the while loop should not break as it should be listening for the response. CompsciOverflow Video lectures on type systems For my job, I need to pick up a working understanding of the implementation of type systems (in particular, how to write typing rules based on a design document). I've been given a copy of Types and Programming Languages and that's great, but I'd really like to watch someone work through it. In my school, the course that teaches this is called "Programming Languages," but searching for "video lectures programming languages" turns up lots of stuff about how to learn C++, which I'm not interested in. I couldn't find anything on MIT Open Courseware. I'd really like video lectures that go through TAPL, but I'll settle for anything useful. Are there good resources out there? StackOverflow Suggestions on a short piece of Haskell code? I'm very new to Haskell and as a way of learning (first project) I am porting code from Tcl to Haskell. The project is a domain specific query language which is translated to SQL queries via a semantic layer. Now, with the operators of the query language restricted in the parser for the language, code similar to this actually seems more silly in Haskell than it did in Tcl :-) type MatchOperator = [Char] getOpJoiner :: MatchOperator -> String getOpJoiner "!=" = " and " getOpJoiner "!~" = " and " getOpJoiner "!~~" = " and " getOpJoiner _ = " or " -- In reality, this can only be "=", "~", "~~" or "==" according to the parser  The code does what is should do (return the appropriate SQL logical joiner of statements depending on the query operator used), but I'm sure it can be made more pretty. CompsciOverflow The stable marriage algorithm with asymmetric arrays I have a question about the stable marriage algorithm, for what I know it can only be used when I have arrays with the same number of elements for building the preference and the ranking matrices. For example if we have 10 students that should be assigned to 10 dorms, then I can build the preference and ranking matrix of this data. The question that I have is what to do in the case that I have, for example, only 5 students to assign and for example 10 dorms. Can I still apply the stable marriage algorithm? Maybe this question is a little bit foolish, but as I said I only saw this algorithm applied to quadratic arrays. QuantOverflow Testing the validity of a factor model for stock returns Consider the following m regression equation system:$$r^i = X^i \beta^i + \epsilon^i \;\;\; \text{for} \;i=1,2,3,..,n$$where r^i is a (T\times 1) vector of the T observations of the dependent variable, X^i is a (T\times k) matrix of independent variables, \beta^i is a (k\times1) vector of the regression coefficients and \epsilon^i is the vector of errors for the T observations of the i^{th} regression. My question is: in order to test the validity of this model for stock returns (i.e. the inclusion of those explanatory variables) using AIC or BIC criterion, should these criterion be computed on a time-series basis (i.e. for each stock), or on a cross-sectional basis (and then averaged over time)? StackOverflow How do i use play ws library in normal sbt project instead of play? When i tried using Play WS library in a normal sbt project instead of play project I was bound to use play.api.Play.current and got java.lang.RuntimeException: "There is no started application" when tried to run the application. Play! framework: define a variable in template? I'm passing to a template an Event object and what I need to do is checking @event.getSeverity value. if the value is positive, I want to color a specific <div> in green. if the value is negative I want to color a specific <div> in red. I couldn't find a way to define a variable. is it possible? it should be I think. anyhow, what's the simplest way accomplishing this? thanks how can i get all row key from column family in cassandra cli? fetch all rowkey from column family in cassandra cli using hector val query=HFactory.createRangesliceQuery(keyspace,le,se,se) query.setColumnFamily("myInfo").setKeys(Integer.MIN_VALUE,Integer.MAX_Value).execute()  OR val query=HFactory.createRangesliceQuery(keyspace,le,se,se) query.setColumnFamily("myInfo").setKeys(Integer.MIN_VALUE,Integer.MAX_Value).setRange("","",true,10).execute()  scala getClass and type erasure? I am a little puzzled by the getClass method when used in generic type case: def a[K](key: K) = { println(key.getClass) }  Shouldn't this always been AnyRef or Any or something? Since type K info is not available in runtime? but a(3) will output Int , and a("fdasf") will output String. Alright, this is a silly question, I get confused by the K and key's type information. CompsciOverflow Can I have a "dependent coproduct type"? I'm reading through the HoTT book and I have a (probably very naive) question about the stuff in the chapter one. The chapter introduces the function type$$ f:A\to B $$and then generalizes it by making B dependent on x:A$$B:A\to\mathcal{U},\qquad g:\prod_{x:A}B(x)$$and that is called the dependent function type. Moving on, the chapter then introduces the product type$$ f:A\times B$$and then generalizes it by making B dependent on x:A$$B:A\to\mathcal{U},\qquad g:\sum_{x:A}B(x)$$and that is called the dependent pair type. I can definitely see a pattern here. Moving on, the chapter then introduces the coproduct type$$ f:A+B$$and ... combobreaker ... there is no discussion of dependent version of this type. Is there some fundamental restriction on that or it is just irrelevant for the topic of the book? In any case can someone help me with intuition on why function and product types? What makes those two so special that they get to be generalized to dependent types and then used to build up everything else? Satisfy edges' constraints when updating node in directed acyclical graph I have a directed acyclical graph. Each node represents an event with start and end dates and each edge represents a constraint between to events with 2 properties: • max interval between previous event end and next event start • min interval between previous event end and next event start When an event's date is updated, all edge constraints should be respected and every other event's date should be recalculated if those constraints are violated. Problem is that there might be multiple conflicting constraints and I'm struggling to find best way to traverse the graph updating events without breaking previously satisfied constraints. I have no formal CS education and my knowledge in this area is quite limited. Is there an existing algorithm to solve my problem? QuantOverflow Which interest rate model for which product Given the multitude of existing interest rate models (ranging from simple to very complex) it would be interesting to know when the additional complexity actually makes sense. The models I have in mind: • Simple one factor (e.g. HW) • Two factor models • Two or one factor models with stochastic volatility • LIBOR-Market, HJM, SABR and SABR-LMM model Are there any rules of thumb to decide which model to use for which product? (Perhaps there is some book dealing with that topic that I am not aware of) Edit 22.02.2014: While trying to answer the question myself I found the follwing very interesting paper on the emperical comparison of interest rate models. Here the authors mainly compare how well the different models can fit market data and hit the relevant market pries after being calibrated. Thus a follow up Question: (that is also related to the question on model validation) Does it suffice to hit the market pries spot on after calibration for a model to qualify for being used in pricing for an instrument ? Or are there other aspects to be considered? (computational speed, statistical fittness, robustness of the hedges)  Let's assume my model fits the market data really well - backtesting however shows that the hedges it provides don't work that well. Also the model might not be able to statistially fit the path of the underlying. E.g. mean reversion can be observed in some markets but not in others. One could argue that risk neutrality does not necessarily entail meaningful real-world scenarios etc. I have good theoretical grasp of the models but have mainly used them for risk management (thus generating paths and analysing what happens to a portfolio or the balance sheet of an enterprise) /r/emacs Temporary key binding Sometime when I do very specific task, I would like to create temporary key-binding so that, during that task, I can easily do repeated functions executed. Once that task is done, restore back old key binding. This keys can be key-chord as well. Is there package available in that use case ? submitted by sudurais [link] [5 comments] Fred Wilson Flurry Yesterday our portfolio company Flurry announced it was being acquired by Yahoo! I thought I’d provide a bit of history since this was an interesting investment for us. Back when Apple was launching its app platform in the winter of 2008, we met with Greg Yardley who had teamed up with Jesse Rohland to build an analytics service for app developers. We had known Greg from his work with Seth Goldstein at Root and we were fans. And it seemed to be a smart idea to give developers the ability to see what people were doing in their mobile apps. So we provided seed financing to Greg and Jesse along with our friends at First Round. Pinch launched the first iOS analytics service and got rapid adoption. But they ran into some challenges, the two primary ones were monetization and getting onto Android and Blackberry (which was relevant back then). And that’s where Flurry entered the picture. Flurry was a pivot into the same business as Pinch was in. They were already on Android and Blackberry but were far behind Pinch on iOS. They were led by a hard charging CEO named Simon Khalaf who had big ideas for monetization. It was a match made in heaven. So the two companies merged and Flurry became the surviving company. Flurry continues to lead the mobile app analytics business. According to Simon’s blog post yesterday, there are 170,000 developers with 542,000 mobile apps using the Flurry service. And now Flurry becomes a Yahoo! branded offering. There is no question that the Flurry data and its advertising products (powered by Flurry’s data) will be a great fit for Yahoo!’s mobile ambitions. So we have a happy ending to a startup story with a few twists and turns. This is an example of where 1+1 equaled a lot more than two. I’ve been involved in a number of “startup mergers”. Some work. Some don’t. This one worked beautifully. StackOverflow Scala path-dependent type: testing for type error [duplicate] This question already has an answer here: I've written some Scala to work with modular arithmetic using path dependent types. The parent class represents a group modulo some integer, and the inner class represents elements of this group. class ModularInt( modulus : Int ) { case class Value( value: Int ) { def plus( a: Value ) : Value = { val rem: Int = (value + a.toInt) % modulus if( rem < 0 ) Value(modulus + rem) else Value(rem) } def toInt = value } }  This seems to work correctly (e.g. modulo 7: 6 + 2 = 1). A nice side effect of using path-dependent types is that numbers modulo different integers will throw a type error if you try to operate them together. However, I would like to insert a test that will confirm this fact. import org.scalatest.FunSuite import org.junit.runner.RunWith import org.scalatest.junit.JUnitRunner @RunWith(classOf[JUnitRunner]) class ModularIntSuite extends FunSuite { test("a mod n + b mod m => error") { val n: Int = 3 val m: Int = 4 val modulen = new ModularInt( n ) val modulem = new ModularInt( m ) val a = new modulen.Value(0) val b = new modulem.Value(0) assert(a plus b) // catch error here } }  However, as a plus b throws a type error at compile time, the test won't even run. Is there anyway to insert a test for a compile error? Or does the fact that compile errors are tests of correctness themselves mean that this is a redundant check (i.e. what I'm trying to do makes no sense)? Thanks Undeadly LibreSSL 2.0.3 Released Bob Beck (beck@) has announced the release of LibreSSL 2.0.3: We have released an update, LibreSSL 2.0.3 - which should be arriving in the LibreSSL directory of an OpenBSD mirror near you very soon. This release includes a number of portability fixes based on the the feedback we have received from the community. It also includes some improvements to the fork detection support. As noted before, we welcome feedback from the broader community. Enjoy, -Bob  QuantOverflow Use of geometric mean for average return of several indices Can anyone give any reference for using the geometric mean to average the returns from several indices? Note, this question is not about the usual use of geometric mean to obtain the average return from a single time series. It is about averaging several indices in a single time step, so for example :- January 2014 index 1 return = 3% index 2 return = 5% index 3 return = -2%  E.g. returns = {3, 5, -2}; meanreturn = (GeometricMean[returns/100. + 1] - 1)*100  Answer: 1.95711 % Edit My current thought as to why the geometric mean might be used to average returns in a single time period is that it produces a lower result than the arithmetic mean, so for generally positive returns with a leptokurtic bias (shown red c/w blue normal dist.) the geometric mean would damp out the contribution of the higher returns. This seems a bit of a kluge though; any references welcome. Averages generated from randomly generated distributed returns around a value of 1% StackOverflow Custom JodaTime serializer using Play Framework's JSON library? How do I implement a custom JodaTime's DateTime serializer/deserializer for JSON? I'm inclined to use the Play Framework's JSON library (2.1.1). There is a default DateTime serializer, but it uses dt.getMillis instead of .toString which would return an ISO compliant String. Writing Reads[T] amd Writes[T] for case classes seems fairly straightforward, but I can't figure out how to do the same for DateTime. QuantOverflow Infinite autocorrelation - Unit root? I have a time series of gold prices, on which I want to build an ARIMA model. The series is autocorrelated and if I can difference as often as I want, it always is. First: data: d1gold Dickey-Fuller = -18.5829, Lag order = 19, p-value = 0.01 alternative hypothesis: stationary Second: data: d2gold Dickey-Fuller = -32.6297, Lag order = 19, p-value = 0.01 alternative hypothesis: stationary .. and so on. What can I do to fit the data in an ARIMA model? Data: https://drive.google.com/file/d/0B7cBu_0IHA17a1lQUlpsS1BJXzg/edit?usp=sharing Best Regards Erik /r/netsec Introduction to computer security Lobsters Crash-only Software QuantOverflow SVCJ (SVJJ) Duffie et. al Model implementation in Matlab I'm attempting to implement aforementioned SVCJ model by Duffie et al in MATLAB. so far without success. It's supposed to price vanilla (european) calls . parameters provided, the expected price is: ~6.0411. The function throws the error: Warning: Reached the limit on the maximum number of intervals in use. Approximate bound on error is 1.1e+03. The integral may not exist, or it may be difficult to approximate numerically to the requested accuracy. Using quadv instead of integral didn't yield the expected result, but instead just returned NaNs.  function price = SVCJDuffie2000Test(S0, K, V0, r, mus, muv, lambda, kappav, ... thetav, sigmav, sigmas, rho, rhoj, q, t, T) % this function should in theory calculate the analytical solution for % the SVCJ model by Duffie et al % % % % % T=0.25 % K = 100; % S0 = 85; % kappav=4; % lambda=4; % rhoj=-0.5; % rho=-0.5; % thetav=0.04; % r=0.05; % sigmas=0.06; % mu = 0.2380 % muv=0.02; % muJ = -0.04; % sigmav=0.1; % V0 = 0.25; % % % mus = log((1+mu)*(1-rhoj*muv))-0.5*(sigmas^2); % SVCJDuffie2000Test( S0, K, V0, r, mus, muv, lambda, kappav, thetav, sigmav, sigmas, rho, rhoj, 0,0,T ) % S0, V0, r, mu, muv, lambda, kappav, % thetav, sigmav, sigmas, rho, rhoj, y = log(S0); X0 = y; % nu, initial vola, nubar long run mean vola nu = V0; c = K; nubar = thetav; sigmav = sigmav; sigmacy = sigmas; mucy = mus; mucv = muv; rhoj = rhoj; rhobar = rho; kappav = kappav; zetabar = q; lambdac = lambda; % specific to SVCJ model lambdabar = lambda; r = r; % not needed for SVJJ, ie SVCJ sigmay = 0; lambday = 0; lambdav = 0; muv = 0; muy = 0; mubar = thetafunc(1,0) - 1 function retval = alphabar(tau,u) thetainter = lambdabar^-1*(lambdac*fc(u,tau)); retval = alpha0(tau, u) - lambdabar*tau*(1+mubar*u)+lambdabar*thetainter; end function retval = betabar(tau,u) a = u*(1-u); b = sigmav*rhobar*u-kappav; gamma = sqrt(b^2+a*sigmav^2); retval = a*(1-exp(-gamma*tau)) / (2*gamma-(gamma+b)*(1-exp(-gamma*tau))); end function retval = alpha0(tau, u) a = u*(1-u); b = sigmav*rhobar*u-kappav; gamma = sqrt(b^2+a*sigmav^2); retval = -r*tau+(r-zetabar)*u*tau-kappav*nubar*... (((gamma+b)/sigmav^2)*tau + ... (2/sigmav^2)*log(1-((gamma+b)/(2*gamma))*(1-exp(-gamma*tau)))); end function retval = fy(u,tau) retval = tau*exp(muy*u+1/2*sigmay^2*u^2); end function retval = fv(u,tau) a = u*(1-u); b = sigmav*rhobar*u-kappav; gamma = sqrt(b^2+a*sigmav^2); c_ = 1 - rhoj*mucv*u; retval = (gamma-b)/((gamma-b)+muv*a)*tau-(2*muv*a)/((gamma*c_)^2-(b*c_-muv*a)^2)... *log(1-((gamma+b)*c_-muv*a)/(2*gamma)*(1-exp(-gamma*tau))); end function retval = fc(u,tau) a = u*(1-u); b = sigmav*rhobar*u-kappav; c_ = 1 - rhoj*mucv*u; gamma = sqrt(b^2+a*sigmav^2); d_ = (gamma-b)/((gamma-b)*c_+mucv*a)*tau-(2*mucv*a)/((gamma*c_)^2-(b*c_-mucv*a)^2)... *log(1-((gamma+b)*c_-mucv*a)/(2*gamma*c_)*(1-exp(-gamma*tau))); retval = exp(mucy*u+sigmacy^2*u^2/2)*d_; end function retval = thetafunc(c1, c2) retval = lambdabar^-1*(lambdac*thetacfunc(c1,c2)); end function retval = thetayfunc(c) retval = exp(muy*c+1/2*sigmay^2*c^2); end function retval = thetavfunc(c) retval = 1/(1-muv*c); end function retval = thetacfunc(c1,c2) retval = exp(mucy*c1+1/2*sigmacy^2*c1^2) / ... (1-mucv*c2-rhoj*mucv*c1); end function retval = psi(y_, u, t, T) retval = exp(alphabar(T-t,u)+u*y_+ betabar(T-t,u)*nu); end % extended transform p.13 function retval = psichi(nu_, u, y, t, T) retval = exp(alphabar(T-t,u)+u*y+ betabar(T-t,u)*nu_); end function retval = Gab(a,b, y, X0_, T) integratefunc = @(nu_) imag(psi(complex(a,nu_*b),X0_, 0, T) * exp(complex(0,-nu_*y))) / nu_; retval = psi(a,X0_,0,T)/2 - 1/pi*integral(integratefunc, 0, Inf, 'ArrayValued',true); %retval = 1; end % depends on payoff function, see p. 6, 18 %aT = -r*T; %bT = 0; %d = 1; % could also be: aT = 0; bT = 1; d = -r*T; GbTplusdminusd = Gab(bT+d,-d,-log(c),X0,T) GbTminusd =Gab(bT,-d,-log(c),X0,T) price = exp(aT)*GbTplusdminusd-c*exp(aT)*GbTminusd; end  Functions in the paper: thetainer: p. 23 alpha0: p.22 alphabar: p.22 betabar: p.22 fy,fv,fc: p. 23 thetayfunc, thetacfunc, thetavfunc: p. 22 psi: p.21 Gab: p. 13 not a function, but depending on the derivative type: GbTplusdminusd,GbTminusd, price: p. 18 lambdabar = lambda, according to: p. 24 StackOverflow Write a function with different return types in OCaml How can I re-write this function in OCaml so that it allows the elements of the tuple to have different types let nth i (x,y,z) = match i with 1->x |2->y |3->z |_->raise (Invalid_argument "nth")  Wondermark Video: Nikola Tesla at KPCC! Here I am playing Tesla at Sunday’s event! My entrance is at 55:00 in the video. Over the weekend I was pleased to participate in a Nikola Tesla event at KPCC, one of our local public radio stations in Los Angeles. Science reporter Sanden Totten (at left in the photo above) hosted a retrospective of Tesla’s life and career, and brought up scientists from the California Science Center and the Griffith Observatory to talk about the principles of electricity. Near the end of the presentation, I got to play a time-traveling Nikola Tesla to do a brief Q&A with Sanden. If you ever get invited to dress up like a famous scientist and tell jokes, I highly recommend it. Here is video of the event on KPCC’s site! Sanden plays Mayor LaGuardia’s posthumous remembrance of Tesla at about 54 minutes in, which precedes my entrance right at 55:00. The event was a co-production with Sanden’s radio show/podcast Brains On, which is a science and education show for kids specifically. There were a lot of kids in the audience at this event and I hope they enjoyed it! I have to thank Sanden for inviting me to participate, and also for providing me with the opportunity to take what should probably be my author photo for time immemorial. BONUS PICTURE: TESLA WITH A TESLA TheoryOverflow Efficient Shamir secret sharing reconstruction Shamir's secret sharing scheme is a well known way to convert a secret into a polynomial and distribute points in this polynomial. Some of these points can then be regrouped to reconstruct the original polynomial and subsequently the secret. The regrouping process can be summarized as finding p(0) in a polynomial knowing only the degree and pairs (x_i, y_i) such that p(x_i) = y_i. My question is: how do I do this efficiently? All implementations I've seen so far are O(n^2) because they recompute the polynomial using Lagrange, with a few optimizations. Are there known faster ways? Wondermark #1048; Looking Forward to Looking Back TheoryOverflow What techniques are there for asking about satisfiability of a first order of a sentence? In reading about SAT in general and applications in first order logic, I've noticed that there are algorithms for answering the question of if a particular instance of a propositional sentence is satisfiable, or if a sentence in a decidable fragment of first order logic is satisfiable (such as Presburger arithmetic) but I haven't seen any algorithms that explicitly attempt to decide satisfiability of first order sentences in general. There are also techniques for answering if a first order sentence is valid, by answering if its negation is unsatisfiable using first order resolution, or tableau methods. It looks to me like first order resolution doesn't terminate for many satisfiable first order sentences. It can tell if a sentence is unsatisfiable but not if a sentence is satisfiable (or just undecidable.) Are there algorithms for asking if a sentence is satisfiable in first order logic? DataTau Live streaming of GraphLab Conference Show DT: Data Science definitions in less than 50 words StackOverflow Invoking a Future inside a receive method and stopping the actor after that I'm creating a bunch of actors that do some work and then they are stopped. In some of these actors I'm invoking 3rd party APIs that return a Future. MyActor extends Actor { .... def receive = { case MsgA => { ... //this evaluates to a Future val p : Future = someFutureAPICall() //stop the actor context stop self } } }  Now since the Future is non-blocking the actor will be stopped right after that (?), even if the Future has not completed. What is the expected behavior in this case ? For example, if I've an onComplete on the Future, will that ever be executed even if the actor has stopped ? MyActor extends Actor { .... def receive = { case MsgA => { ... //this evaluates to a Future val p : Future = someFutureAPICall() p.onComplete { //will these expressions ever be evaluated ? case Success(x) => log.info("Success") case Failure(f) => log.info("Failure") } //stop the actor context stop self } } }  Planet Theory Shifts In Algorithm Design How to find approximate page rank fast, among other things Jennifer Chayes is the current director of a research lab in Cambridge—that is Cambridge Massachusetts—for a company called Microsoft. She is famous for her own work in many areas of theory, including phase transitions of complex systems. She is also famous for her ability to create and manage research groups, which is a rare and wonderful skill. Today Ken and I wish to talk about how to be “shifty” in algorithm design. There is nothing underhanded, but it’s a different playing field from what we grew up with. Ken first noted the paradigm shift in 1984 when hearing Leslie Valiant’s inaugural talk on “Probably Approximately Correct” learning at the Royal Society in London. Last year Leslie wrote a book with this title. That’s about learning, while here we want to discuss the effect on algorithm design. We illustrate this for a paper (ArXiv version) authored by Chayes and Christian Borgs, Michael Brautbar, and Shang-Hua Teng. It is titled, “Multi-Scale Matrix Sampling and Sublinear-Time PageRank Computation.” Since PageRank is a vital application, they don’t want their algorithms to be galactic. They trade off by not having the algorithms “solve” the problems the way we used to. In The Good Old Days I, Dick, recall the “good old days of theory.” When I first started working in theory—a sort of double meaning—I could only use deterministic methods. I needed to get the exact answer, no approximations. I had to solve the problem that I was given—no changing the problem. Well sometimes I did that, but mostly I had to solve the problem that was presented to me. In the good old days of theory, we got a problem, we worked on it, and sometimes we solved it. Nothing shifty, no changing the problem or modifying the goal. I actually like today better than the “good old days,” so I do not romanticize them. One way to explain the notion of the good old days is to quote from a Monty Python skit about four Yorkshiremen talking about the good old days. We pick it up a few lines after one of them says, “I was happier then and I had nothin’. We used to live in this tiny old house with great big holes in the roof.” … ${\mathsf{FIRST \ YORKSHIREMAN}}$: You were lucky. We lived for three months in a paper bag in a septic tank. We used to have to get up at six in the morning, clean the paper bag, eat a crust of stale bread, go to work down t’ mill, fourteen hours a day, week-in week-out, for sixpence a week, and when we got home our Dad would thrash us to sleep wi’ his belt. ${\mathsf{SECOND \ YORKSHIREMAN}}$: Luxury. We used to have to get out of the lake at six o’clock in the morning, clean the lake, eat a handful of ‘ot gravel, work twenty hour day at mill for tuppence a month, come home, and Dad would thrash us to sleep with a broken bottle, if we were lucky! ${\mathsf{THIRD \ YORKSHIREMAN}}$: Well, of course, we had it tough. We used to ‘ave to get up out of shoebox at twelve o’clock at night and lick road clean wit’ tongue. We had two bits of cold gravel, worked twenty-four hours a day at mill for sixpence every four years, and when we got home our Dad would slice us in two wit’ bread knife. ${\mathsf{FOURTH \ YORKSHIREMAN}}$: Right. I had to get up in the morning at ten o’clock at night half an hour before I went to bed, drink a cup of sulphuric acid, work twenty-nine hours a day down mill, and pay mill owner for permission to come to work, and when we got home, our Dad and our mother would kill us and dance about on our graves singing Hallelujah. ${\mathsf{FIRST \ YORKSHIREMAN}}$: And you try and tell the young people of today that ….. they won’t believe you. ${\mathsf{ALL}}$: They won’t! Today Those were the days. I did feel like I sometimes worked twenty-nine hours a day, I was paid by Yale, but so little that perhaps it felt like I paid them. Never had to drink a cup of sulphuric acid—but the coffee—oh you get the idea. Now today, in the 21st century, we have a better way to attack problems. We change the problem, often to one that is more tractable and useful. In many situations solving the exact problem is not really what a practitioner needs. If computing X exactly requires too much time, then it is useless to compute it. A perfect example is the weather: computing tomorrow’s weather in a week’s time is clearly not very useful. The brilliance of the current approach is that we can change the problem. There are at least two major ways to do this: ${\bullet }$ Change the answer required. Allow approximation, or allow a partial answer. Do not insist on an exact answer. ${ \bullet }$ Change the algorithmic method. Allow algorithms that can be wrong, or allow algorithms that use randomness. Do not insist that the algorithm is a perfect deterministic one. This is exactly what Chayes and her co-authors have done. So let’s take a look at what they do in their paper. PageRank In their paper they study PageRank, which is the definition and algorithm made famous by Google. It gives a way to rank webpages in response to a query that supplements criteria from the query itself. An old query-specific criterion was the number of matches to a keyword in the query. Rather than rank solely by this count, PageRank emphasizes a general page score. The score is sometimes interpreted as a measure of “popularity” or “authority,” leading to the following circular-seeming definitions: ${\bullet}$ A webpage is popular if it has a healthy number of links from popular pages. ${\bullet}$ A webpage is authoritative if it is well cited, especially by other authoritative pages. What the PageRank score actually denotes mathematically is the likelihood that a person randomly traversing links will arrive at any particular page. This includes a frequency ${\alpha}$ with which the person will stop clicking, do something healthier like ride a bicycle, and start again on a “random” webpage. The situation can be modeled by the classic random walk on a directed graph. We have a graph ${G}$ on ${N}$ nodes and an ${N \times N}$ matrix ${M}$ that is row-stochastic, meaning the entries in each row are non-negative and sum to ${1}$. Given that the web-walker is at node ${i}$, the entry ${M[i,j]}$ is the probability of going next to node ${j}$. If node ${i}$ has out-degree ${b}$, then $\displaystyle M[i,j] = \frac{\alpha}{N} + \begin{cases} (1 - \alpha)\frac{1}{b} & \text{if~} i \text{~links to~} j\\ 0 & \text{otherwise.}\end{cases}$ We can tweak this e.g. by modeling the user hitting the “Back” button on the browser, or jumping to another browser tab, or using a search engine. We could also set ${\alpha}$ higher in case page ${i}$ has few or no outgoing links. We still get an ${M}$, and since the use of ${\alpha}$ effectively makes the graph strongly connected and averts certain pathologies, we get a beautiful conclusion from random-walk theory: There is a unique stationary distribution ${P}$, which is the unique left-eigenvector for the largest eigenvalue, which as normalized above is ${1}$: $\displaystyle P M = P.$ Then the PageRank of node ${i}$ is ${P(i)}$. It is remarkable that this simple, salient idea from the good old days works so well. A further fact from the theory (and use of ${\alpha}$) is that if you start at any node, in the long run you will find yourself on page ${i}$ with frequency ${P(i)}$. Here is Wikipedia’s graphical example: The issue is: how to compute ${P(i)}$? In the good old days this was a trivial problem—just use linear algebra. But now the issue is that ${N}$ is really big, let alone ${N \times N}$ being unspeakably big. The ${N}$ is too big even to get an approximation via the “further fact,” that is by simulating a random walk on the whole graph, and classical sparse-matrix methods might only help a little. This is where Chayes and company change the game: let us care about computing ${P(i)}$ only for some ${i}$‘s, and even then, let us be content with fairly rough approximation. The Shifted Problems The approximation to PageRank is called SignificantPageRank. The paper gives a randomized algorithm that solves the following problem. Let us be given a graph. Then, given a target threshold ${\Delta}$ and an approximation factor ${c > 1}$, we are asked to output a set ${S}$ of nodes such that with high probability, ${S}$ contains all nodes of PageRank at least ${\Delta}$, and no node of PageRank smaller than ${\frac{1}{c}\Delta}$. This is a perfect example of the shift. The algorithm is random, and the problem is to find not the nodes with a given PageRank, but those that are not too far away. The nifty point is that the algorithm can tolerate fuzzing the matrix ${M}$, in a manner called SARA for “sparse and approximate row access”: Given ${i}$ and ${\epsilon}$, return a set ${S}$ of ${O(1/\epsilon)}$ columns ${j}$ and values ${p_j}$ such that for all ${j}$: • ${j \in S \implies |p_j - M[i,j]| \leq \epsilon}$, and • ${j \notin S \implies M[i,j] \leq \epsilon}$. It is important to use this for different values of ${\epsilon}$. The cost of a query ${(i,\epsilon) \rightarrow S}$ is ${\Theta(|S|) = O(1/\epsilon)}$. If we picture “${N}$” as “exponential” and take ${\epsilon = 1/\mathsf{poly}(n)}$ where ${n = \log N}$, then this becomes an approximative version of ${M}$ being succinct, which we just talked about. In this scaling of ${\epsilon}$ we are effectively limiting to a ${\mathsf{poly}(n)}$ local portion ${S}$ of the graph around node ${i}$. Since we also have ${\frac{\alpha}{N} \ll \epsilon}$, under SARA entries outside ${S}$ would become effectively zero, so that the chance of “teleporting” outside ${S}$ on the whole would be regarded as negligible. In fact the paper also researches the case where each Web user always starts afresh at a “home node” ${u}$ in that portion, making ${M[i,u] = \alpha}$ just for that user. Then the ${\alpha}$-related probability is not negligible, and the resulting user-dependent estimate is called PersonalizedPageRank. The problem they need to solve for SignificantPageRank then becomes “SignificantColumnSums”: Given ${M}$ and ${\Delta,c}$ as above, find a set ${S}$ of columns such that for all columns ${j}$: • ${\sum_i M[i,j] \geq \Delta \implies j \in S}$; • ${\sum_i M[i,j] \leq \frac{1}{c}\Delta \implies j \notin S}$. An even simpler problem which they use as a stepping-stone is “VectorSum”: Given a length-${N}$ vector ${P}$ with entries in ${[0,1]}$, and ${1 \leq \Delta \leq N}$ and ${c > 1}$: • output yes if ${{}\text{~}\sum_i P(i) \geq \Delta}$; • output no if ${{}\text{~}\sum_i P(i) \leq \frac{\Delta}{c}}$, don’t-care otherwise. The goal is always to avoid looking at all ${N}$ nodes or entries, but only an ${N^{1 - \gamma}}$ or so portion of them, where ${\gamma > 0}$. Thus the problem shift is necessitated by ${N}$ being huge. This isn’t my good-old-days idea of solving a problem, but can be called “${(\Delta,c)}$-solving” it. Multi-Level Sampling and Results Ken and I are interested because these are similar to problems we have been encountering in our quest for more cases where quantum algorithms can be simulated in classical randomized polynomial time. Thus any new ideas are appreciated, and what catches our eye is a multi-level approximation scheme that exploits the requirement of SARA to work for different ${\epsilon}$. The situations are different, but we hope to adapt them. The situation for VectorSum is that a probe ${(i,\epsilon) \rightarrow P(i)}$ still costs order-of ${1/\epsilon}$, and returns ${0}$ unless ${P(i) \geq \epsilon}$. A simple-minded use of a set ${S}$ of random probes for the same ${\epsilon}$ would yield the estimate $\displaystyle \frac{N}{|S|} \sum_{i \in S: P(i) \geq \epsilon} P(i).$ The resulting error has order ${N\epsilon}$, so we need ${\epsilon \approx \frac{\Delta}{N}}$ which is rather demanding. Indeed the total cost would have order ${\frac{|S|N}{\Delta}}$, where ${S}$ needs to be so large as to kill any hope of making the cost ${N^{1 - \gamma}}$ or even ${o(N)}$. In the pivotal case where ${\sum_i P(i) = \Delta}$, we would need ${|S| \approx \frac{N}{\Delta}}$, incurring cost on the order of ${(\frac{N}{\Delta})^2}$. However, they show that by using a different precision ${\epsilon_t \approx \frac{t}{|S|}}$ for each random probe, they can get acceptable error with a reasonably small number of probes. The case ${t = 1}$ where we have ${\epsilon \approx \frac{1}{|S|} = \frac{\Delta}{N}}$ occurs only once, so its ${\frac{N}{\Delta}}$ cost can be tolerated. Other probes have smaller cost, and while their precisions are looser, the aggregate precision on the estimate becomes good enough for the following result: Theorem 1 Given ${P,c,\Delta}$ as above and ${\delta > 0}$, VectorSum can be ${(\Delta,c)}$-solved with probability at least ${1 - \delta}$ and cost $\displaystyle O\left(\frac{N}{\Delta}\left(\frac{1}{c-1}\right)^2 \log\left(\frac{N}{\Delta(c-1)}\right)\log\left(\frac{2}{\delta}\right)\right).$ Well in the good old days before LaTeX we wouldn’t even have been easily able to write such a formula with a typewriter, let alone prove it. But it is certainly better than ${(\frac{N}{\Delta})^2}$, and allows taking ${\Delta = N^{\gamma}}$ to meet the goal of runtime ${\tilde{O}(N^{1 - \gamma})}$. As usual, for the details on SignificantColumnSums and the application problems, see the paper. Open Problems Do you miss the good old days? Or do you like the current approaches? What shifts, what new types of changing the goals, might we see in the future? For clearly today will one day be the “good old days.” [changed qualifier on "certain pathologies", may as well footnote here that the "Back" button creates a Markov Chain with memory.] /r/clojure Clojure in the Large UnixOverflow Do I need to setup my ZFS (using FreeBSD 10) for foreign-language files (e.g. filenames in Japanese/Thai/etc language)? I am in the process of setting up a ZFS file server. I am planning to store all sorts of files in it. I have several hundred music tracks that are in Japanese/Korean native character. I want the file server to retain this native character naming. Do I need to setup my ZFS pool with utf8only and/or normalization options? e.g. copy japanese songs into file server (via mounted shared folder in Windows). If I view the files on my file server in Windows Explorer, they must be displayed in their native characters. Thanks a lot. QuantOverflow How is the price of a bond actually determined? How the price of a bond is actually determined? Is it the supply-demand that determines the price first and then the YTM is calculated on the back of this for that bond. Or is it that the changes to interest rate curve comes first and then the bond is priced using the typical discounting method and that becomes the price in stock market? CompsciOverflow Regular expression (ab U a)* to NFA with two states (Sipser)? In the 3rd edition of Sipser's Introduction to the Theory of Computation (example 1.56, p.68), there is a step-by-step procedure to transform (ab U a)* into a NFA. And then the text ends with: "In this example, the procedure gives an NFA with eight states, but the smallest equivalent NFA has only two states. Can you find it?" Nope. I can't. After a good deal of head scratching, I've convinced myself that it's not doable. But being a novice, I'm probably wrong. Can anyone help? Thanks! /r/netsec HTML5 Canvas Fingerprinting /r/freebsd Manic malware Mayhem spreads through Linux, FreeBSD web servers StackOverflow unable to run oowriter as web user I have a web server set up in my laptop. I was building a web application which i am accessing in the local network. I have a php file which contains following line: command = "/usr/bin/oowriter --invisible --convert-to pdf /var/www/hackathon/document/gaurav.doc -outdir /var/www/hackathon/pdf/" exec(command,output,return_var); print_r(output); So, the problem is when i run in terminal above php runs perfect.But when accessed via web browser, output is just empty and no conver is done as intended. apache error log shows following lines: [Java framework] Error in function createSettingsDocument (elements.cxx). javaldx failed! Warning: failed to read path from javaldx i have tried solutions https://wiki.archlinux.org/index.php/Libreoffice#Fixing_Java_Framework_Error . but it didnt work. i am having OpenJdk 7. Anybody having any idea how to make this work?? CompsciOverflow PROBLEM IN NUMBER SYSTEM [migrated] how there are two different conversion results for the same bases. Am i doing something wrong ! Please help ! thanks in advance. /r/netsec The talk about de-anonymizing Tor at the BlackHat conference has been removed /r/compsci Several new papers by Bob Harper and others Lobsters FsCheck + XUnit = The Bomb /r/netsec Extensively Adaptable Sploits and Tools for Encroaching on Router Security. (Holcomb, J. et al. HOPE X 19 July 2014 slides) The Codist: Everything Is Insecure StackOverflow Is there a way to flatten nested monads of different types? I am not sure how to describe this problem, so I'll just show the type signatures. I have an instance of the following: val x:Future[F[Future[F[B]]]] = ???  And I want an instance of: val y:Future[F[B]] = ???  F is a Monad, so I have the following methods: def pure[A](a:A):F[A] = ??? def flatMap[A, B](fa:F[A], f:A => F[B]):F[B] = ??? def map[A, B](fa:F[A], f:A => B):F[B] = flatMap(fa, (a:A) => pure(f(a)))  I think the following should work, but it does not feel right: x.flatMap { fWithFuture => val p = Promise[F[B]] flatMap(fWithFuture, (futureF: Future[F[B]]) => { p.completeWith(futureF) pure(()) }) p.future }  Is there a concept I'm missing? A bit of background information. I am trying to define a function like this: def flatMap[A, B](fa:Future[F[A]], f: A => Future[F[B]]):Future[F[B]] = ???  Maybe this is conceptually a weird thing. Any tips on useful abstractions are welcome. Lobsters 1 second per second is harder than it sounds StackOverflow Why's Clojure failing from JNLP now? I've got a Java app that's been in use for awhile and uses embedded Clojure scripts for some of the calculations. Old versions of it still run fine with the current JRE, but if I go to build a new version (only changes to the code are in unrelated Java code) it fails to launch properly. It also runs perfectly fine locally. Only JNLP is affected. java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessDeclaredMembers") at java.security.AccessControlContext.checkPermission(Unknown Source) at java.security.AccessController.checkPermission(Unknown Source) at java.lang.SecurityManager.checkPermission(Unknown Source) at java.lang.Class.checkMemberAccess(Unknown Source) at java.lang.Class.getDeclaredConstructors(Unknown Source) at clojure.coregenerate_proxy.invoke(core_proxy.clj:137) at clojure.coreget_proxy_class.doInvoke(core_proxy.clj:261) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.coreapply.invoke(core.clj:617) at clojure.coreproxy.doInvoke(core_proxy.clj:335) at clojure.lang.RestFn.invoke(RestFn.java:1152) at clojure.lang.Var.invoke(Var.java:465) at clojure.lang.AFn.applyToHelper(AFn.java:248) at clojure.lang.Var.applyTo(Var.java:532) at clojure.lang.Compiler.macroexpand1(Compiler.java:6468) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6546) at clojure.lang.Compiler.analyze(Compiler.java:6361) at clojure.lang.Compiler.analyze(Compiler.java:6322) at clojure.lang.CompilerBodyExprParser.parse(Compiler.java:5708) at clojure.lang.CompilerLetExprParser.parse(Compiler.java:6009) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6560) at clojure.lang.Compiler.analyze(Compiler.java:6361) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548) at clojure.lang.Compiler.analyze(Compiler.java:6361) at clojure.lang.Compiler.analyze(Compiler.java:6322) at clojure.lang.CompilerBodyExprParser.parse(Compiler.java:5708) at clojure.lang.CompilerFnMethod.parse(Compiler.java:5139) at clojure.lang.CompilerFnExpr.parse(Compiler.java:3751) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6558) at clojure.lang.Compiler.analyze(Compiler.java:6361) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548) at clojure.lang.Compiler.analyze(Compiler.java:6361) at clojure.lang.Compiler.access100(Compiler.java:37) at clojure.lang.CompilerDefExprParser.parse(Compiler.java:529) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6560) at clojure.lang.Compiler.analyze(Compiler.java:6361) at clojure.lang.Compiler.analyze(Compiler.java:6322) at clojure.lang.Compiler.eval(Compiler.java:6623) at clojure.lang.Compiler.load(Compiler.java:7064) at clojure.lang.RT.loadResourceScript(RT.java:370) at clojure.lang.RT.loadResourceScript(RT.java:361) at clojure.lang.RT.load(RT.java:440) at clojure.lang.RT.load(RT.java:411) at clojure.coreloadfn__1451.invoke(core.clj:5530) at clojure.coreload.doInvoke(core.clj:5529) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.coreeval1658.invoke(core.clj:6154) at clojure.lang.Compiler.eval(Compiler.java:6619) at clojure.lang.Compiler.load(Compiler.java:7064) at clojure.lang.RT.loadResourceScript(RT.java:370) at clojure.lang.RT.loadResourceScript(RT.java:361) at clojure.lang.RT.load(RT.java:440) at clojure.lang.RT.load(RT.java:411) at clojure.lang.RT.doInit(RT.java:447) at clojure.lang.RT.<clinit>(RT.java:329) at myapp(myapp.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.sun.javaws.Launcher.executeApplication(Unknown Source) at com.sun.javaws.Launcher.executeMainClass(Unknown Source) at com.sun.javaws.Launcher.doLaunchApp(Unknown Source) at com.sun.javaws.Launcher.run(Unknown Source) at java.lang.Thread.run(Unknown Source)  The line that it's failing on is this: RT.loadResourceScript("myapp/clojure_scripts.clj"); // Initialize Clojure script processor with our script  We're currently using a self-signed cert for this app (I've been working with Comodo for the last week trying to get a trusted cert to see if that helps, but I've still not received it and I need this app working ASAP). What's odd to me is that old versions (also self-signed the exact same way) work. I just can't build a new version. I've rolled back my configs so that only actual java code changes should be different, but still no luck. I'm missing something somewhere... ccw clojure set up troubles I have been working with eclipse + counterclockwise for clojure development, using only the default template for non-web based work. However, I would now like to connect my environment to my localhost apache server. I began a new project, and changed the template from default to clojure-app and it did give me a web based framework in the project.clj file, as follows: (defproject guestbook "0.1.0-SNAPSHOT" :description "FIXME: write description" :url "http://example.com/FIXME" :dependencies [[org.clojure/clojure "1.6.0"] [compojure "1.1.6"] [hiccup "1.0.5"] [ring-server "0.3.1"]] :plugins [[lein-ring "0.8.10"]] :ring {:handler guestbook.handler/app :init guestbook.handler/init :destroy guestbook.handler/destroy} :aot :all :profiles {:production {:ring {:open-browser? true, :stacktraces? false, :auto-reload? false}} :dev {:dependencies [[ring-mock "0.1.5"] [ring/ring-devel "1.2.1"]]}})  Next, I try to set up ring through lein ring server in the terminal. But I get this error: "'ring' is not a task. See 'lein help'." I am not sure how to proceed, or exactly what I can do to get the localhost server up. CompsciOverflow Call by need compared to pass by function SML uses pass‑by‑value, Haskell uses call‑by‑need. Unless I'm wrong (the purpose of this question), one can do call‑by‑need with SML, passing a function instead of a value; a function to be later evaluated when needed. Adapting a classical example: (* This will print 0 *) fun pass_by_function (cond, a1, a2) = if cond then a1 else a2 val e1 = fn () => 1 * 0 val e2 = fn () => 1 div 0 val _ = print (Int.toString (pass_by_function (true, e1, e2) ())) (* This will fail with a divide error *) fun pass_by_value (cond, a1, a2) = if cond then a1 else a2 val _ = print (Int.toString (pass_by_value (true, 1 * 0, 1 div 0)))  It's a big matter for Haskell to be call‑by‑need. I'm just wondering if there is something else or if it's just, say “syntactic sugar”, compared to the way it needs to be done with SML. Is this just a different way to write the same or is there a semantic difference I'm not aware of? Passing a function, is it indeed always the same as call‑by‑need? If not, when and how does it differs? StackOverflow How to log all requests with unfiltered I'm using unfiltered to provide restful API, and have defined several intents. Now I have a new requirement, I have to log all the request url into a file, but I can't find a good solution for it. I've read the document of unfiltered, not found something like "filter/interceptor" in SpringMVC. Is there any way to do it? arXiv Logic in Computer Science Weihrauch degrees of finding equilibria in sequential games. (arXiv:1407.5587v1 [cs.LO]) Various operations related to infinite sequential games are classified in the Weihrauch lattice. Differentially Private MIMO Filtering for Event Streams and Spatio-Temporal Monitoring. (arXiv:1407.5553v1 [cs.SY]) Many large-scale systems such as intelligent transportation systems, smart grids or smart buildings collect data about the activities of their users to optimize their operations. In a typical scenario, signals originate from many sensors capturing events involving these users, and several statistics of interest need to be continuously published in real-time. In addition, in order to encourage user participation, privacy issues need to be taken into consideration. This paper considers the problem of providing differential privacy guarantees for such multi-input multi-output systems operating continuously. We show in particular how to construct various extensions of the zero-forcing equalization mechanism, which we previously proposed for single-input single-output systems. We also describe an application to privately monitoring and forecasting occupancy in a building equipped with a dense network of motion detection sensors, which is useful for example to control its HVAC system. Tractable Model for Rate in Self-Backhauled Millimeter Wave Cellular Networks. (arXiv:1407.5537v1 [cs.IT]) Millimeter wave (mmW) cellular systems will require high gain directional antennas and dense base station (BS) deployments to overcome high near field path loss and poor diffraction. As a desirable side effect, high gain antennas provide interference isolation, providing an opportunity to incorporate self-backhauling--BSs backhauling among themselves in a mesh architecture without significant loss in throughput--to enable the requisite large BS densities. The use of directional antennas and resource sharing between access and backhaul links leads to coverage and rate trends that differ significantly from conventional microwave (\muW) cellular systems. In this paper, we propose a general and tractable mmW cellular model capturing these key trends and characterize the associated rate distribution. The developed model and analysis is validated using actual building locations from dense urban settings and empirically-derived path loss models. The analysis shows that in sharp contrast to the interference limited nature of \muW cellular networks, the spectral efficiency of mmW networks (besides total rate) also increases with BS density particularly at the cell edge. Increasing the system bandwidth, although boosting median and peak rates, does not significantly influence the cell edge rate. With self-backhauling, different combinations of the wired backhaul fraction (i.e. the faction of BSs with a wired connection) and BS density are shown to guarantee the same median rate (QoS). Process-Oriented Parallel Programming with an Application to Data-Intensive Computing. (arXiv:1407.5524v1 [cs.PL]) We introduce process-oriented programming as a natural extension of object-oriented programming for parallel computing. It is based on the observation that every class of an object-oriented language can be instantiated as a process, accessible via a remote pointer. The introduction of process pointers requires no syntax extension, identifies processes with programming objects, and enables processes to exchange information simply by executing remote methods. Process-oriented programming is a high-level language alternative to multithreading, MPI and many other languages, environments and tools currently used for parallel computations. It implements natural object-based parallelism using only minimal syntax extension of existing languages, such as C++ and Python, and has therefore the potential to lead to widespread adoption of parallel programming. We implemented a prototype system for running processes using C++ with MPI and used it to compute a large three-dimensional Fourier transform on a computer cluster built of commodity hardware components. Three-dimensional Fourier transform is a prototype of a data-intensive application with a complex data-access pattern. The process-oriented code is only a few hundred lines long, and attains very high data throughput by achieving massive parallelism and maximizing hardware utilization. L(2,1)-labelling of Circular-arc Graph. (arXiv:1407.5488v1 [cs.DM]) An L(2,1)-labelling of a graph G=(V, E) is \lambda_{2,1}(G) a function f from the vertex set V (G) to the set of non-negative integers such that adjacent vertices get numbers at least two apart, and vertices at distance two get distinct numbers. The L(2,1)-labelling number denoted by \lambda_{2,1}(G) of G is the minimum range of labels over all such labelling. In this article, it is shown that, for a circular-arc graph G, the upper bound of \lambda_{2,1}(G) is \Delta+3\omega, where \Delta and \omega represents the maximum degree of the vertices and size of maximum clique respectively. Joint Channel Selection and Power Control in Infrastructureless Wireless Networks: A Multi-Player Multi-Armed Bandit Framework. (arXiv:1407.5447v1 [cs.GT]) This paper deals with the problem of efficient resource allocation in dynamic infrastructureless wireless networks. Assuming a reactive interference-limited scenario, each transmitter is allowed to select one frequency channel (from a common pool) together with a power level at each transmission trial; hence, for all transmitters, not only the fading gain, but also the number of interfering transmissions and their transmit powers are varying over time. Due to the absence of a central controller and time-varying network characteristics, it is highly inefficient for transmitters to acquire global channel and network knowledge. Therefore a reasonable assumption is that transmitters have no knowledge of fading gains, interference, and network topology. Each transmitting node selfishly aims at maximizing its average reward (or minimizing its average cost), which is a function of the action of that specific transmitter as well as those of all other transmitters. This scenario is modeled as a multi-player multi-armed adversarial bandit game, in which multiple players receive an a priori unknown reward with an arbitrarily time-varying distribution by sequentially pulling an arm, selected from a known and finite set of arms. Since players do not know the arm with the highest average reward in advance, they attempt to minimize their so-called regret, determined by the set of players' actions, while attempting to achieve equilibrium in some sense. To this end, we design in this paper two joint power level and channel selection strategies. We prove that the gap between the average reward achieved by our approaches and that based on the best fixed strategy converges to zero asymptotically. Moreover, the empirical joint frequencies of the game converge to the set of correlated equilibria. We further characterize this set for two special cases of our designed game. Composing security protocols: from confidentiality to privacy. (arXiv:1407.5444v1 [cs.CR]) Security protocols are used in many of our daily-life applications, and our privacy largely depends on their design. Formal verification techniques have proved their usefulness to analyse these protocols, but they become so complex that modular techniques have to be developed. We propose several results to safely compose security protocols. We consider arbitrary primitives modeled using an equational theory, and a rich process algebra close to the applied pi calculus. Relying on these composition results, we are able to derive some security properties on a protocol from the security analysis performed on each sub-protocol individually. We consider parallel composition and the case of key-exchange protocols. Our results apply to deal with confidentiality but also privacy-type properties (e.g. anonymity, unlinkability) expressed using a notion of equivalence. We illustrate the usefulness of our composition results on protocols from the 3G phone application. Cooperative Game Theoretic Solution Concepts for top-k Problems. (arXiv:1407.5442v1 [cs.SI]) The problem of finding the k most critical nodes, referred to as the top\text{-}k problem, is a very important one in several contexts such as information diffusion and preference aggregation in social networks, clustering of data points, etc. It has been observed in the literature that the value allotted to a node by most of the popular cooperative game theoretic solution concepts, acts as a good measure of appropriateness of that node (or a data point) to be included in the top\text{-}k set, by itself. However, in general, nodes having the highest k values are not the desirable top\text{-}k nodes, because the appropriateness of a node to be a part of the top\text{-}k set depends on other nodes in the set. As this is not explicitly captured by cooperative game theoretic solution concepts, it is necessary to post-process the obtained values in order to output the suitable top\text{-}k nodes. In this paper, we propose several such post-processing methods and give reasoning behind each of them, and also propose a standalone algorithm that combines cooperative game theoretic solution concepts with the popular greedy hill-climbing algorithm. Privacy-Preserving Important Passage Retrieval. (arXiv:1407.5416v1 [cs.IR]) State-of-the-art important passage retrieval methods obtain very good results, but do not take into account privacy issues. In this paper, we present a privacy preserving method that relies on creating secure representations of documents. Our approach allows for third parties to retrieve important passages from documents without learning anything regarding their content. We use a hashing scheme known as Secure Binary Embeddings to convert a key phrase and bag-of-words representation to bit strings in a way that allows the computation of approximate distances, instead of exact ones. Experiments show that our secure system yield similar results to its non-private counterpart on both clean text and noisy speech recognized text. Gateless Treasure: How to Get Sensitive Information from Unprotected External Storage on Android Phones. (arXiv:1407.5410v1 [cs.CR]) The flexibility of Android is mainly based on the cross application (app for short) access mechanism. Aside from providing convenience for both app developers and users, such a mechanism also brings the data in external storage which originally regarded harmless into serious privacy breaches. In this paper, we studied various popular apps and developed three different attacks leveraging the public information freely available to zero-permission apps. The first attack can smuggle out the sensitive data which reveals the victim's profile information directly, such as real name, phone number, email address, social networking accounts, etc. With the help of image comparing techniques and current social networking services, we can find out the owners of the phone numbers acquired from a victim's WhatsApp with a high probability. Moreover, the location of the victim is also not spared in the disaster. Comparing to the previous location inferring attacks, our approach is more general since the victim's identity is also acquired by our attack. These three attacks reveal the privacy risks of external storage are much more serious than people previously thought and need to be addressed urgently. In fact, all these threats are caused by the file system used by external storage which calls into question the reasonability of the assumptions on the cross app access mechanism. To the end, we propose a preliminary mitigation approach to achieve a delicate balance between utility and privacy of the data stored in external storage. Model based design of super schedulers managing catastrophic scenario in hard real time systems. (arXiv:1407.5404v1 [cs.DC]) The conventional design of real-time approaches depends heavily on the normal performance of systems and it often becomes incapacitated in dealing with catastrophic scenarios effectively. There are several investigations carried out to effectively tackle large scale catastrophe of a plant and how real-time systems must reorganize itself to respond optimally to changing scenarios to reduce catastrophe and aid human intervention. The study presented here is in this direction and the model accommodates catastrophe generated tasks while it tries to minimize the total number of deadline miss, by dynamically scheduling the unusual pattern of tasks. The problem is NP hard. We prove the methods for an optimal scheduling algorithm. We also derive a model to maintain the stability of the processes. Moreover, we study the problem of minimizing the number of processors required for scheduling with a set of periodic and sporadic hard real time tasks with primary/backup mechanism to achieve fault tolerance. EDF scheduling algorithms are used on each processor to manage scenario changes. Finally we present a simulation of super scheduler with small, medium and large real time tasks pattern for catastrophe management. Low-Effort Specification Debugging and Analysis. (arXiv:1407.5399v1 [cs.SE]) Reactive synthesis deals with the automated construction of implementations of reactive systems from their specifications. To make the approach feasible in practice, systems engineers need effective and efficient means of debugging these specifications. In this paper, we provide techniques for report-based specification debugging, wherein salient properties of a specification are analyzed, and the result presented to the user in the form of a report. This provides a low-effort way to debug specifications, complementing high-effort techniques including the simulation of synthesized implementations. We demonstrate the usefulness of our report-based specification debugging toolkit by providing examples in the context of generalized reactivity(1) synthesis. Are There Good Mistakes? A Theoretical Analysis of CEGIS. (arXiv:1407.5397v1 [cs.LO]) Counterexample-guided inductive synthesis CEGIS is used to synthesize programs from a candidate space of programs. The technique is guaranteed to terminate and synthesize the correct program if the space of candidate programs is finite. But the technique may or may not terminate with the correct program if the candidate space of programs is infinite. In this paper, we perform a theoretical analysis of counterexample-guided inductive synthesis technique. We investigate whether the set of candidate spaces for which the correct program can be synthesized using CEGIS depends on the counterexamples used in inductive synthesis, that is, whether there are good mistakes which would increase the synthesis power. We investigate whether the use of minimal counterexamples instead of arbitrary counterexamples expands the set of candidate spaces of programs for which inductive synthesis can successfully synthesize a correct program. We consider two kinds of counterexamples: minimal counterexamples and history bounded counterexamples. The history bounded counterexample used in any iteration of CEGIS is bounded by the examples used in previous iterations of inductive synthesis. We examine the relative change in power of inductive synthesis in both cases. We show that the synthesis technique using minimal counterexamples MinCEGIS has the same synthesis power as CEGIS but the synthesis technique using history bounded counterexamples HCEGIS has different power than that of CEGIS, but none dominates the other. How to Handle Assumptions in Synthesis. (arXiv:1407.5395v1 [cs.LO]) The increased interest in reactive synthesis over the last decade has led to many improved solutions but also to many new questions. In this paper, we discuss the question of how to deal with assumptions on environment behavior. We present four goals that we think should be met and review several different possibilities that have been proposed. We argue that each of them falls short in at least one aspect. Program Synthesis and Linear Operator Semantics. (arXiv:1407.5393v1 [cs.PL]) For deterministic and probabilistic programs we investigate the problem of program synthesis and program optimisation (with respect to non-functional properties) in the general setting of global optimisation. This approach is based on the representation of the semantics of programs and program fragments in terms of linear operators, i.e. as matrices. We exploit in particular the fact that we can automatically generate the representation of the semantics of elementary blocks. These can then can be used in order to compositionally assemble the semantics of a whole program, i.e. the generator of the corresponding Discrete Time Markov Chain (DTMC). We also utilise a generalised version of Abstract Interpretation suitable for this linear algebraic or functional analytical framework in order to formulate semantical constraints (invariants) and optimisation objectives (for example performance requirements). Synthesis of a simple self-stabilizing system. (arXiv:1407.5392v1 [cs.LO]) With the increasing importance of distributed systems as a computing paradigm, a systematic approach to their design is needed. Although the area of formal verification has made enormous advances towards this goal, the resulting functionalities are limited to detecting problems in a particular design. By means of a classical example, we illustrate a simple template-based approach to computer-aided design of distributed systems based on leveraging the well-known technique of bounded model checking to the synthesis setting. Nontransitive Ranking to Enhance Routing Decision in MANETS. (arXiv:1407.5385v1 [cs.NI]) An ad hoc network is an infrastructureless network in which nodes perform terminal as well as routing functions. A routing protocol is the only substitute to complete the communications in the absence of an access point. In spite of that mobile nodes or so called routers uses some mechanism for calculating the best route when it has multiple routes for the same destination. On the basis of one or more metrics routes are ranked from best to worst. But, in an ad hoc network many factors can affect this decision, such as the delay, load, route lifetime etc. Thus measuring and finding routes on the basis of crisp mathematical model for all these attributes is complicated. That why, the fuzzy approach for best route determination is required for MANET because some of the metrics are fuzzy or vague and the classical ranking of routes and transitivity in the ranking does not hold. The proposed Nontransitive Route Ranking subjective comparison of one route with others and performs nontransitive ranking to rank routes from best to worst. The pairwise comparisons of each route with others give more accurate and fair comparison. The proposed ranking is easier than classical ranking in which metrics have assigned some value and these values are combined to obtain the ranking. Experimental result shows the efficiency of the proposed model. Keywords: Fuzzy, Rank, Nontransitive, Route, Ranking, Relativity The Intractability of Dynamic Mechanism Design. (arXiv:1407.5373v1 [cs.GT]) We introduce a dynamic mechanism design problem in which the designer wants to offer for sale an item to an agent, and another item to the same agent at some point in the future. The agent's joint distribution of valuations for the two items is known, and the agent knows the valuation for the current item (but not for the one in the future). The designer seeks to maximize expected revenue, and the auction must be deterministic, truthful, and ex post individually rational. The optimum mechanism involves a protocol whereby the seller elicits the buyer's current valuation, and based on the bid makes two take-it-or-leave-it offers, one for now and one for the future. We show that finding the optimum mechanism --- arguably the simplest meaningful dynamic mechanism design problem imaginable --- is NP-hard. We also prove several positive results, among them a polynomial linear programming-based algorithm for the optimum randomized auction, and we show strong separations in revenue between non-adaptive, adaptive, and randomized auctions. A Comparative Analysis for Determining the Optimal Path using PSO and GA. (arXiv:1407.5327v1 [cs.NI]) Significant research has been carried out recently to find the optimal path in network routing. Among them, the evolutionary algorithm approach is an area where work is carried out extensively. We in this paper have used particle swarm optimization (PSO) and genetic algorithm (GA) for finding the optimal path and the concept of region based network is introduced along with the use of indirect encoding. We demonstrate the advantage of fitness value and hop count in both PSO and GA. A comparative study of PSO and genetic algorithm (GA) is carried out, and it was found that PSO converged to arrive at the optimal path much faster than GA. An Optimum Scheduling Approach for Creating Optimal Priority of Jobs with Business Values in Cloud Computing. (arXiv:1407.5320v1 [cs.DC]) Realizing an optimal task scheduling by taking into account the business importance of jobs has become a matter of interest in pay and use model of Cloud computing. Introduction of an appropriate model for an efficient task scheduling technique could derive benefit to the service providers as well as clients. In this paper, we have addressed two major challenges which has implications on the performance of the Cloud system. One of the major issues is handling technical aspects of distributing the tasks for targeted gains and the second issue is related to the handling of the business priority for concurrently resolving business complexity related to cloud consumers. A coordinated scheduling can be achieved by considering the weightage of both aspects viz. technical requirements and business requirements appropriately. It can be done in such a way that it meets the QoS requirements of technical domain as well as business domain. Along with the technical priority a business Bp is required in creating a resultant priority which could be given to stages of further processing, like task allocation and arbitration schemes. Here we consider a technical priority Tp that is governed by a semi-adaptive scheduling algorithm whereas the resultant priority is derived in which a Business Priority Bp layer encapsulates the Technical Priority Tp to achieve the overall priority of the incoming tasks. It results in a Hybrid priority creation, which is a combination of both technical priority Tp and business priority Bp. By taking into account the business priority of the jobs it is possible to achieve a higher service level satisfaction for the tasks which are submitted with their native technical priority. With this approach the waiting time of the tasks tends to get reduced and it gives a better service level satisfaction. The Expressive Power of Epistemic \mu-Calculus. (arXiv:1407.5166v1 [cs.LO]) While the \mu-calculus notoriously subsumes Alternating-time Temporal Logic (ATL), we show that the epistemic \mu-calculus does not subsume ATL with imperfect information (ATL_i) for the synchronous perfect-recall semantics. To prove this we first establish that jumping parity tree automata (JTA), a recently introduced extension of alternating parity tree automata, are expressively equivalent to the epistemic \mu-calculus, and this for any knowledge semantics. Using this result we also show that, for bounded-memory semantics, the epistemic \mu-calculus is not more expressive than the standard \mu-calculus, and that its satisfiability problem is EXPTIME-complete. Cooperative oligopoly games with boundedly rational firms. (arXiv:1202.3885v2 [cs.GT] UPDATED) We analyze cooperative Cournot games with boundedly rational firms. Due to cogni- tive constraints, the members of a coalition cannot accurately predict the coalitional structure of the non-members. Thus, they compute their value using simple heuris- tics. In particular, they assign various non-equilibrium probability distributions over the outsiders' set of partitions. We construct the characteristic function of a coalition in such an environment and we analyze the core of the corresponding games. We show that the core is non-empty provided the number of firms in the market is sufficiently large. Moreover, we show that if two distributions over the set of partitions are related via first-order dominance, then the core of the game under the dominated distribution is a subset of the core under the dominant distribution. StackOverflow Why is cmis returning an error on datalist item creation in alfresco? I have the following piece of code  try { dataList.asInstanceOf[AlfrescoFolder].createItem( Map("cmis:objectTypeId" -> "D:dl:contact", "cmis:name" -> (data.emailAddress + "_Newsletter_sub"), "dl:contactEmail" -> data.emailAddress) ) } catch{ case e: CmisRuntimeException => Logger.error("\n\n ====> silently ignoring this exception because the item is created\n"+e.getMessage+"\n\n"); }  which works fine and creates the data list item. but it throws an exception with the message Execution exception[[CmisRuntimeException: Newly created object is not an item! New id: Object Id: I'd like to know if this a normal/known issue with this api method and why it's happening. Thanks DragonFly BSD Digest mrsas(4) made default for Thunderbolt RAID As mentioned before, the mrsas(4) driver works best for ‘Thunderbolt’ RAID controllers. Now, the switch has happened. USB tethering on DragonFly Tethering now works via the (Updated for correct attribution) StackOverflow Scala -- transform and map data The raw data file I will be reading is a tab-delimited and one of the fields is a timestamp: timestamp userId keyword 1405377264 A google 1405378945 B yahoo 1405377264 C facebook  I got a case class defined as: case class Event(date: String, userId: Int, keyword: String)  How do I go about transforming the timestamp to Date format and then map to the Event case class? I have the logic to convert the timestamp to Date: import java.text.SimpleDateFormat import java.util.Date val df = new SimpleDateFormat("yyyy-MM-dd") val dt = new Date(timestamp*1000L) val date = df.format(dt)  What is the right way to convert the raw data and map it to the case class? Thanks! DataTau Leo Breiman - Statistical Modeling: The Two Cultures Stanford Large Network Dataset Collection Planet Clojure July 2014 London Clojure Dojo at ThoughtWorks When: Tuesday, July 29, 2014 from 7:00 PM to 9:30 PM (BST) Where: ThoughtWorks London Office 173 High Holborn WC1V London United Kingdom Hosted By: London Clojurians Register for this event now at : http://www.eventbrite.com/e/july-2014-london-clojure-dojo-at-thoughtworks-tickets-12365027129?aff=rss Event Details: London Clojure Dojo at ThoughtWorks The goal of the session is to help people learn to start working with Clojure through practical exercises, but we hope that more experienced developers will also come along to help form a bit of a London Clojure community. The dojo is a great place for new and experienced clojure coders to learn more. If you want to know how to run your own dojo or get an idea of what dojos are like you can read more here. We hope to break up into groups for the dojo. So if you have a laptop with a working clojure environment please bring it along. We’ll be discussing the meetup on the london-clojurians mailing list Clojure is a JVM language that has syntactically similarities to Lisp, full integration with Java and its libraries and focuses on providing a solution to the issue of single machine concurrency. Its small core makes it surprisingly easy for Java developers to pick up and it provides a powerful set of concurrency strategies and data structures designed to make immutable data easy to work with. If you went to Rich Hickey’s LJC talk about creating Clojure you’ll already know this, if not it’s well worth watching the Rich Hickey “Clojure for Java Programmers” video or Stuart Halloway “Radical Simplicity” video . TheoryOverflow Can some body help me to understand this Recommendations Engine Came across this recommendation engine document. Can't understand it. K-means clustering , Apriori StackOverflow What features does Pedestal have compared to other Clojure web frameworks? Recently Relevance announced Pedestal, a new web framework for Clojure. How does this framework compare to other Clojure web frameworks such as Compojure, Webjure, Conjure, Moustache and so on, in terms of features, ease of application development, developer tools, etc. Planet Theory Classification of Passes in Football Matches using Spatiotemporal Data Authors: Michael Horton, Joachim Gudmundsson, Sanjay Chawla, Joël Estephan Download: PDF Abstract: A knowledgeable observer of a game of football (soccer) can make a subjective evaluation of the quality of passes made between players during the game. We investigate the problem of producing an automated system to make the same evaluation of passes. We present a model that constructs numerical predictor variables from spatiotemporal match data using feature functions based on methods from computational geometry, and then learns a classification function from labelled examples of the predictor variables. Furthermore, the learned classifiers are analysed to determine if there is a relationship between the complexity of the algorithm that computed the predictor variable and the importance of the variable to the classifier. Experimental results show that we are able to produce a classifier with 85.8% accuracy on classifying passes as Good, OK or Bad, and that the predictor variables computed using complex methods from computational geometry are of moderate importance to the learned classifiers. Finally, we show that the inter-rater agreement on pass classification between the machine classifier and a human observer is of similar magnitude to the agreement between two observers. Symblicit algorithms for optimal strategy synthesis in monotonic Markov decision processes Authors: Aaron Bohy, Véronique Bruyère, Jean-François Raskin Download: PDF Abstract: When treating Markov decision processes (MDPs) with large state spaces, using explicit representations quickly becomes unfeasible. Lately, Wimmer et al. have proposed a so-called symblicit algorithm for the synthesis of optimal strategies in MDPs, in the quantitative setting of expected mean-payoff. This algorithm, based on the strategy iteration algorithm of Howard and Veinott, efficiently combines symbolic and explicit data structures, and uses binary decision diagrams as symbolic representation. The aim of this paper is to show that the new data structure of pseudo-antichains (an extension of antichains) provides another interesting alternative, especially for the class of monotonic MDPs. We design efficient pseudo-antichain based symblicit algorithms (with open source implementations) for two quantitative settings: the expected mean-payoff and the stochastic shortest path. For two practical applications coming from automated planning and LTL synthesis, we report promising experimental results w.r.t. both the run time and the memory consumption. Complexity of Grundy coloring and its variants Authors: Edouard Bonnet, Florent Foucaud, Eun Jung Kim, Florian Sikora Download: PDF Abstract: The Grundy number of a graph is the maximum number of colors used by the greedy coloring algorithm over all vertex orderings. In this paper, we study the computational complexity of Grundy Coloring, the problem of determining whether a given graph has Grundy number at least k. We show that Grundy Coloring can be solved in time O^*(2^n) on graphs of order n. While the problem is known to be solvable in time f(k,w)n for graphs of treewidth w, we prove that under the Exponential Time Hypothesis, it cannot be computed in time O^*(c^w), for every constant c. We also consider two previously studied variants of Grundy Coloring, namely Weak Grundy Coloring and Connected Grundy Coloring. We show that Weak Grundy Coloring is fixed-parameter tractable with respect to the weak Grundy number. In stark contrast, it turns out that checking whether a given graph has connected Grundy number at least k is NP-complete already for k=7. On the Algorithmic Lov\'{a}sz Local Lemma Authors: Ioannis Giotis, Lefteris Kirousis, Kostas I. Psaromiligkos, Dimitrios M. Thilikos Download: PDF Abstract: The algorithm for Lov\'{a}sz Local Lemma by Moser and Tardos gives a constructive way to prove the existence of combinatorial objects that satisfy a system of constraints. We present an alternative and simpler analysis of the algorithm that does not involve counting witness-trees. Design and Analysis of RS Sort Authors: Harsh Ranjan, Sumit Agarwal Download: PDF Abstract: This paper introduces a new comparison base stable sorting algorithm, named RS sort. RS Sort involves only the comparison of pair of elements in an array which ultimately sorts the array and does not involve the comparison of each element with every other element. RS sort tries to build upon the relationship established between the elements in each pass. Suppose there is an array containing three elements a1, a2, a3 and if a relationship exist such that a1<a2 and a2<a3 then it can be established that a1<a3 and so there is no need to compare a1 and a3. Sorting is a fundamental operation in computer science. RS sort is analyzed both theoretically and empirically. We have performed its Empirical analysis and compared its performance with the well-known quick sort for various input types. How the Experts Algorithm Can Help Solve LPs Online Authors: Anupam Gupta, Marco Molinaro Download: PDF Abstract: We consider the problem of solving packing/covering LPs online, when the columns of the constraint matrix are presented in random order. This problem has received much attention: the main open question is to figure out how large the right-hand sides of the LPs have to be (compared to the entries on the left-hand side of the constraint) to get (1+\epsilon)-approximations online? It is known that the RHS has to be \Omega(\epsilon^{-2} \log m) times the left-hand sides, where m is the number of constraints. In this paper we give a primal-dual algorithm to achieve this bound for all packing LPs, and also for a class of mixed packing/covering LPs. Our algorithms construct dual solutions using a regret-minimizing online learning algorithm in a black-box fashion, and use them to construct primal solutions. The adversarial guarantee that holds for the constructed duals help us to take care of most of the correlations that arise in the algorithm; the remaining correlations are handled via martingale concentration and maximal inequalities. These ideas lead to conceptually simple and modular algorithms, which we hope will be useful in other contexts. Efficient Algorithms for the Closest Pair Problem and Applications Authors: Sanguthevar Rajasekaran, Sudipta Pathak Download: PDF Abstract: The closest pair problem (CPP) is one of the well studied and fundamental problems in computing. Given a set of points in a metric space, the problem is to identify the pair of closest points. Another closely related problem is the fixed radius nearest neighbors problem (FRNNP). Given a set of points and a radius R, the problem is, for every input point p, to identify all the other input points that are within a distance of R from p. A naive deterministic algorithm can solve these problems in quadratic time. CPP as well as FRNNP play a vital role in computational biology, computational finance, share market analysis, weather prediction, entomology, electro cardiograph, N-body simulations, molecular simulations, etc. As a result, any improvements made in solving CPP and FRNNP will have immediate implications for the solution of numerous problems in these domains. We live in an era of big data and processing these data take large amounts of time. Speeding up data processing algorithms is thus much more essential now than ever before. In this paper we present algorithms for CPP and FRNNP that improve (in theory and/or practice) the best-known algorithms reported in the literature for CPP and FRNNP. These algorithms also improve the best-known algorithms for related applications including time series motif mining and the two locus problem in Genome Wide Association Studies (GWAS). Hellinger volume and number-on-the-forehead communication complexity Authors: Troy Lee, Nikos Leonardos, Michael Saks, Fengming Wang Download: PDF Abstract: Information-theoretic methods have proven to be a very powerful tool in communication complexity, in particular giving an elegant proof of the linear lower bound for the two-party disjointness function, and tight lower bounds on disjointness in the multi-party number-in-the-hand (NIH) model. In this paper, we study the applicability of information theoretic methods to the multi-party number-on-the-forehead model (NOF), where determining the complexity of disjointness remains an important open problem. There are two basic parts to the NIH disjointness lower bound: a direct sum theorem and a lower bound on the one-bit AND function using a beautiful connection between Hellinger distance and protocols revealed by Bar-Yossef, Jayram, Kumar and Sivakumar [BYJKS04]. Inspired by this connection, we introduce the notion of Hellinger volume. We show that it lower bounds the information cost of multi-party NOF protocols and provide a small toolbox that allows one to manipulate several Hellinger volume terms and lower bound a Hellinger volume when the distributions involved satisfy certain conditions. In doing so, we prove a new upper bound on the difference between the arithmetic mean and the geometric mean in terms of relative entropy. We then apply these new tools to obtain a lower bound on the informational complexity of the AND_k function in the NOF setting. Finally, we discuss the difficulties of proving a direct sum theorem for information cost in the NOF model. Lower Bounds on the Oracle Complexity of Nonsmooth Convex Optimization via Information Theory Authors: Gábor Braun, Cristóbal Guzmán, Sebastian Pokutta Download: PDF Abstract: We present an information-theoretic approach to lower bound the oracle complexity of nonsmooth black box convex optimization, unifying previous lower bounding techniques by identifying a combinatorial problem, namely string guessing, as a single source of hardness. As a measure of complexity we use distributional oracle complexity, which subsumes randomized oracle complexity as well as worst-case oracle complexity. We obtain strong lower bounds on distributional oracle complexity for the box [-1,1]^n, as well as for the L^p-ball for p \geq 1 (for both low-scale and large-scale regimes), matching worst-case lower bounds, and hence we close the gap between distributional complexity, and in particular, randomized complexity, and worst-case complexity. Furthermore, the bounds remain essentially the same for high-probability and bounded-error oracle complexity, and even for combination of the two, i.e., bounded-error high-probability oracle complexity. This considerably extends the applicability of known bounds. A Sane Proof that COLk \le COL3 Authors: William Gasarch Download: PDF Abstract: Let COLk be the set of all k-colorable graphs. It is easy to show that if a<b then COLa \le COLb (poly time reduction). Using the Cook-Levin theorem it is easy to show that if 3 \le a< b then COLb \le COLa. However this proof is insane in that it translates a graph to a formula and then the formula to a graph. We give a simple proof that COLk \le COL3. StackOverflow SessionStore in ring From ring's source, SessionStore requires read-session to return an empty map if the key is not found: (read-session [store key] "Read a session map from the store. If the key is not found, an empty map is returned.") But in the memory namespace in MemoryStore it inplements read-session deferring the atom which returns nil if the key is not found. Is there a reason not to implement the full spec of the protocol, or is it a missing feature? /r/compsci thoughts about how to properly credit software and algorithms Lobsters Fireside chat with Jiahua Chen, creator of Gogs (Go Git Service) and Macaron Lambda the Ultimate Forum wither formal methods? I sometimes get a bee in my bonnet to look for tools to do model driven design and the like. So, I found a list of verification and synthesis tools. The mind boggles. For little people such as myself, I wish there were a table somewhere that listed the range of features and then showed what each system did/not support. I want stuff that would help with application development: user interface and state machine (e.g. audio playback engine + ui controlling it) type modeling and code generation (and round tripping). Anybody know of such guidance for the laymen? Portland Pattern Repository Nested Functions (by redbac8.lnk.telstra.net 20 hours ago) HN Daily Daily Hacker News for 2014-07-21 July 21, 2014 Planet Theory Using finite automata to draw graphs The diagram below describes a finite state machine that takes as input a description of an indifference graph, and produces as output a 1-planar drawing of it (that is, a drawing with each edge crossed at most once). Indifference graphs are the graphs that can be constructed by the following process. Initialize an active set of vertices to be the empty set, and then perform a sequence of steps of two types: either add a new vertex to the active set and make it adjacent to all previous active vertices, or inactivate a vertex (removing it from the active set but not from the graph). Thus, they can be represented by a sequence of binary values that specify whether the next step is to add or inactivate a vertex. These values are the input to the finite state machine. In a paper at WADS last year with Bannister and Cabello, we showed how to test 1-planarity for several special classes of graphs, but not for indifference graphs. Some of our algorithms involved proving the existence of a finite set of forbidden configurations, and that idea works here, too: an indifference graph turns out to be 1-planar if and only if, for every K6 subgraph, the first three vertices of the subgraph (in the activation order) have no later neighbors outside the subgraph, and the last three vertices have no other earlier neighbors. K6 is 1-planar, but it has essentially only one drawing (modulo permutation of the vertices), and any example of this configuration would have a seventh vertex connected to four of the K6 vertices, something that's not possible in a 1-planar drawing. At one level, the state diagram above can be viewed as a diagram for detecting this forbidden configuration. Every right-going transition is one that adds an active vertex, and every left-going transition is one that removes an active vertex. If a transition of either type does not exist in the diagram, it means that a step of that type will lead to an inescapable failure state. But the only missing transitions are the ones that would create a six-vertex active set by a sequence of transitions that does not end in three consecutive right arrows (creating a K6 in which one of the last three vertices has an earlier neighbor) or the ones that would exit a six-vertex active set by a sequence of transitions that does not begin with three consecutive left arrows (creating a K6 in which one of the first three vertices has a later neighbor). So, this automaton recognizes only the graphs that have no forbidden configuration. On another level, the drawings within each state of the diagram show how to use this finite state machine to construct a drawing. Each state is labeled with a drawing of its active vertices, possibly with a yellow region that represents earlier inactive parts of the drawing that can no longer be modified. The numbers on the vertices give the order in which they were activated. For each left transition, it is always possible to remove the oldest active vertex from the drawing and replace the parts of the drawing surrounding it by a yellow region to create a drawing that matches the new state. Similarly, for each right transition, it is always possible to add one more active vertex to the drawing, connect it to the other active vertices, and then simplify some parts of the drawing to yellow regions, again creating a drawing that matches the new state. Therefore, every graph that can be recognized by this state diagram has a 1-planar drawing. Since the machine described by the diagram finds a drawing for a given indifference graph if and only if the graph has no forbidden configurations, it follows that these forbidden configurations are the only ones we need to describe the 1-planar graphs and that this machine correctly finds a 1-planar drawing for every indifference graph that has one. This same technique doesn't always generalize: A result from my WADS paper that it's NP-complete to test 1-planarity for graphs of bounded bandwidth shows that, even when a class of graphs can be represented by strings of symbols from a finite alphabet it's not always going to be possible to find a finite state machine to test 1-planarity. But it would be interesting to find more graph classes for which the same simple technique works. StackOverflow Scala ListViewSelectionChange event outputs unexpected range I'm currently trying to use a scala.swing.ListView in a Scala application. I basically want to get the selected item of a ListView, as soon as the selection changes. Here a sample which can be pasted into the Scala prompt. import scala.swing._ class MainView extends MainFrame { import ListView._ val selector = new ListView(List[Int]()) { renderer = Renderer(x => x.toString) selection.reactions += { case event.ListSelectionChanged(source, range, _) => if(!range.isEmpty) { println("selection " + range) } } selection.intervalMode = IntervalMode.Single } selector.listData = 1 +: selector.listData selector.listData = 2 +: selector.listData selector.listData = 3 +: selector.listData selector.listData = 4 +: selector.listData selector.listData = 5 +: selector.listData contents = new GridBagPanel(){ layout(new ScrollPane(selector)) = new Constraints() { gridx = 0 gridy = 0 weighty = 1 fill = GridBagPanel.Fill.Vertical } } visible = true } var view = new MainView()  If I would now select the first element in the list, I would expect Range to contain only 0, if I then select the second element, I would expect only 1 and so on. However, if I select the first and then the second element, Range contains (0, 1). If I select the fifth and then the third element, range contains (2, 3, 4). So, range contains always the range between the last and the current selected item, in ascending order. Sadly, the Scala documentation does not state how this API works. Does anyone know what I'm doing wrong here? Might this be the wrong event for reacting on ListView selection changes? /r/netsec Network Forensics Puzzle (with Walkthrough) [x-post r/blackhat] /r/compsci The Imitation Game - trailer for upcoming movie about Alan Turing and the Enigma machine. CS & Maths Hey guys, I'm gonna be doing a joint honours cs and maths bsc in the UK soon and was just wondering if I've bitten off more than I can chew here. First off I have very little programming experience but schools I applied to seemed to stress that you didn't need any although I have been trying to learn python over the summer on codecademy and have some LateX and HTML background (I know it doesn't count)... On the math side of things I'm not THAT worried as I just got a 5 on Calc AB and according to my uni syllabus, the first semester maths is basically review for me along with some calc 2, vectors/matrices and ofc discrete maths. My main motivation for doing cs AND math is that I really enjoyed calc and would like to finish it off along with the fact that the degree is pretty flexible. Anyone think I'll survive if I put the work in? If a degree like this is attractive to employers? Any input, opinions or advice would be much appreciated! Thanks!! submitted by pboy07 [link] [7 comments] /r/clojure a Leiningen plugin using the asset-minifier StackOverflow Spark ClassNotFoundException running the master I have downloaded and built Spark 0.80 using sbt/sbt assembly. It was successful. However when running ./bin/start-master.sh the following error is seen in the log file Spark Command: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java -cp :/shared/spark-0.8.0-incubating-bin-hadoop1/conf:/shared/spark-0.8.0-incubating-bin-hadoop1/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop1.0.4.jar /shared/spark-0.8.0-incubating-bin-hadoop1/assembly/target/scala-2.9.3/spark-assembly_2.9.3-0.8.0-incubating-hadoop1.0.4.jar -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.master.Master --ip mellyrn.local --port 7077 --webui-port 8080 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/deploy/master/Master Caused by: java.lang.ClassNotFoundException: org.apache.spark.deploy.master.Master at java.net.URLClassLoader1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.LauncherAppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)  Update: after doing sbt clean (per suggestion below) it is running: see screenshot. Planet FreeBSD Application awareness of receive side scaling (RSS) on FreeBSD Part of testing this receive side scaling work is designing a set of APIs that allow for some kind of affinity awareness. It's not easy - the general case is difficult and highly varying. But something has to be tested! So, where should it begin? The main tricky part of this is the difference between incoming, outgoing and listening sockets. For incoming traffic, the NIC has already calculated the RSS hash value and there's already a map between RSS hash and destination CPU. Well, destination queue to be much more precise; then there's a CPU for that queue. For outgoing traffic, the thread(s) in question can be scheduled on any CPU core and as you have more cores, it's increasingly unlikely to be the right one. In FreeBSD, the default is to direct dispatch transmit related socket and protocol work in the thread that started it, save a handful of places like TCP timers. Once the driver if_transmit() method is called to transmit a frame it can check the mbuf to see what the flowid is and map that to a destination transmit queue. Before RSS, that's typically done to keep packets vaguely in some semblance of in-order behaviour - ie, for a given traffic flow between two endpoints (say, IP, or TCP, or UDP) the packets should be transmitted in-order. It wasn't really done for CPU affinity reasons. Before RSS, there was no real consistency with how drivers hashed traffic upon receive, nor any rules on how it should select an outbound transmit queue for a given buffer. Most multi-queue drivers got it "mostly right". They definitely didn't try to make any CPU affinity choices - it was all done to preserve the in-order behaviour of traffic flows. For an incoming socket, all the information about the destination CPU can be calculated from the RSS hash provided during frame reception. So, for TCP, the RSS hash for the received ACK during the three way handshake goes into the inpcb entry. For UDP it's not so simple (and the inpcb doesn't get a hash entry for UDP - I'll explain why below.) For an outgoing socket, all the information about the eventual destination CPU isn't necessarily available. If the application knows the source/destination IP and source/destination port then it (or the kernel) can calculate the RSS hash that the hardware would calculate upon frame reception and use that to populate the inpcb. However this isn't typically known - frequently the source IP and port won't be explicitly defined and it'll be up to the kernel to choose them for the application. So, during socket creation, the destination CPU can't be known. So to make it simple (and to make it simple for me to ensure the driver and protocol stack parts are working right) my focus has been on incoming sockets and incoming packets, rather than trying to handle outgoing sockets. I can handle outbound sockets easily enough - I just need to do a software hash calculation once all of the required information is available (ie, the source IP and port is selected) and populate the inpcb with that particular value. But I decided to not have to try and debug that at the same time as I debugged the driver side and the protocol stack side, so it's a "later" task. For TCP, traffic for a given connection will use the same source/destination IP and source/destination port values. So for a given socket, it'll always hash to the same value. However, for UDP, it's quite possible to get UDP traffic from a variety of different source IP/ports and respond from a variety of different source/IP ports. This means that the RSS hash value that we can store in the inpcb isn't at all guaranteed to be the same for all subsequent socket writes. Ok, so given all of that above information, how exactly is this supposed to work? Well, the slightly more interesting and pressing problem is how to break out incoming requests/packets to multiple receive threads. In traditional UNIX socket setups, there are a couple of common design patterns for farming off incoming requests to multiple worker threads: • There's one thread that just does accept() (for TCP) or recv() (for UDP) and it then farms off new connections to userland worker threads; or • There are multiple userland worker threads which all wait on a single socket for accept() or recv() - and hope that the OS will only wake up one thread to hand work to. It turns out that the OS may wake up one thread at a time for accept() or recv() but then userland threads will sit in a loop trying to accept connections / packets - and then you tend to find they get called a lot only to find another worker thread that was running stole the workload. Oops. I decided this wasn't really acceptable for the RSS work. I needed a way to redirect traffic to a thread that's also pinned to the same CPU as the receive RSS bucket. I decided the cheapest way would be to allow multiple PCB entries for the same socket details (eg, multiple TCP sockets listening on *:80). Since the PCBGROUPS code in this instance has one PCB hash per RSS bucket, all I had to do was to teach the stack that wildcard listen PCB entries (eg, *:80) could also exist in each PCB hash bucket and to use those in preference to the global PCB hash. The idea behind this decision is pretty simple - Robert Watson already did all this great work in setting up and debugging PCBGROUPS and then made the RSS work leverage that. All I'd have to do is to have one userland thread in each RSS bucket and have the listen socket for that thread be in the RSS bucket. Then any incoming packet would first check the PCBGROUP that matched the RSS bucket indicated by the RSS hash from the hardware - and it'd find the "right" PCB entry in the "right" PCBGROUP PCB has table for the "right" RSS bucket. That's what I did for both TCP and UDP. So the programming model is thus: • First, query the RSS sysctl (net.inet.rss) for the RSS configuration - this gives the number of RSS buckets and the RSS bucket -> CPU mapping. • Then create one worker thread per RSS bucket.. • .. and pin each thread to the indicated CPU. • Next, each worker thread creates one listen socket.. • .. sets the IP_BINDANY or IP6_BINDANY option to indicate that there'll be multiple RSS entries bound to the given listen details (eg, binding to *:80); • .. then IP_RSS_LISTEN_BUCKET to set which RSS bucket the incoming socket should live in; • Then for UDP - call bind() • Or for TCP - call bind(), then call listen() Each worker thread will then receive TCP connections / UDP frames that are local to that CPU. Writing data out the TCP socket will also stay local to that CPU. Writing UDP frames out doesn't - and I'm about to cover that. Yes, it's annoying because now you're not just able to choose an IO model that's convenient for your application / coding style. Oops. Ok, so what's up with UDP? The problem with UDP is that outbound responses may be to an arbitrary destination setup and thus may actually be considered "local" to another CPU. Most common services don't do this - they'll send the UDP response to the same remote IP and port that it was sent from. My plan for UDP (and TCP in some instances, see below!) is four-fold: • When receiving UDP frames, optionally mark them with RSS hash and flowid information. • When transmitting UDP frames, allow userspace to inform the kernel about a pre-calculated RSS hash / flow information. • For the fully-connected setup (ie, where a single socket is connect() ed to a given UDP remote IP:port and frame exchange only occurs between the fixed IP and port details) - cache the RSS flow information in the inpcb; • .. and for all other situations (if it's not connected, if there's no hint from userland, if it's going to a destination that isn't in the inpcb) - just do a software hash calculation on the outgoing details. I mostly have the the first two UDP options implemented (ie, where userland caches the information to re-use when transmitting the response) and I'll commit them to FreeBSD soon. The other two options are the "correct" way to do the default methods but it'll take some time to get right. Ok, so does it work? I don't have graphs. Mostly because I'm slack. I'll do up some before I present this - likely at BSDCan 2015. My testing has been done with Intel 1G and 10G NICs on desktop Ivy Bridge 4-core hardware. So yes, server class hardware will behave better. For incoming TCP workloads (eg a webserver) then yes, there's no lock contention between CPUs in the NIC driver or network stack any longer. The main lock contention between CPUs is the VM and allocator paths. If you're doing disk IO then that'll also show up. For incoming UDP workloads, I've seen it scale linearly on 10G NICs (ixgbe(4)) from one to four cores. This is with no-defragmentation, 510 byte sized datagrams. Ie, 1 core reception (ie, all flows to one core) was ~ 250,000 pps into userland with just straight UDP reception and no flow/hash information via recvmsg(); 135,000 pps into userland with UDP reception and flow/hash information via recvmsg(). 4 core reception was ~ 1.1 million pps into userland, roughly ~ 255,000 pps per core. There's no contention between CPU cores at all. Unfortunately what I was sending was markedly different. The driver quite happily received 1.1 million frames on one queue and up to 2.1 million when all four queues were busy. So there's definitely room for improvement. Now, there is lock contention - it's just not between CPU cores. Now that I'm getting past the between-core contention, we see the within-core contention. For TCP HTTP request reception and bulk response transmission, most of the contention I'm currently seeing is between the driver transmit paths. So, the following occurs: • TCP stack writes some data out; • NIC if_transmit() method is called; • It tries to grab the queue lock and succeeds; It then appends the frame to the buf_ring and schedules a transmit out the NIC. This bit is fine. But then whilst the transmit lock is held, because the driver is taking frames from the buf_ring to push into the NIC TX DMA queue • The NIC queue interrupt fires, scheduling the software interrupt thread; • This pre-empts the existing running transmit thread; • The NIC code tries to grab the transmit lock to handle completed transmissions; • .. and it fails, because the code it preempted holds the transmit lock already. So there's some context switching and thrashing going on there which needs to be addressed. Ok, what about UDP? It turns out there's some lock contention with the socket receive buffer. The soreceive_dgram() routine grabs the socket receive buffer (SOCKBUF_LOCK()) to see if there's anything to return. If not, and if it can sleep, it'll call sbwait() that will release the lock and msleep() waiting for the protocol stack to indicate that something has been received. However, since we're receiving packets at such a very high rate, it seems that the receive protocol path contends with the socket buffer lock that is held by the userland code trying to receive a datagram. It pre-empts the user thread, tries to grab the lock and fails - and then goes to sleep until the userland code finishes with the lock. soreceive_dgram() doesn't hold the lock for very long - but I do see upwards of a million context switches a second. To wrap up - I'm pleased with how things are going. I've found and fixed some issues with the igb(4) and ixgbe(4) drivers that were partly my fault and the traffic is now quite happily and correctly being processed in parallel. There are issues with scaling within a core that are now being exposed and I'm glad to say I'm going to ignore them for now and focus on wrapping up what I've started. There's a bunch more to talk about and I'm going to do it in follow-up posts. • what I'm going to do about UDP transmit in more detail; • what about creating outbound connections and how applications can be structured to handle this; • handling IP fragments and rehashing packets to be mostly in-order - and what happens when we can't guarantee ordering with the hardware hashing UDP frames to a 4-tuple; • CPU hash rebalancing - what if a specific bucket gets too much CPU load for some reason; • randomly creating a toeplitz RSS hash key at bootup and how that should be verified; • multi-socket CPU and IO domain awareness; • .. and whatever else I'm going to stumble across whilst I'm slowly fleshing this stuff out. I hope to get the UDP transmit side of things completed in the next couple of weeks so I can teach memcached about TCP and UDP RSS. After that, who knows! CompsciOverflow Non-Approximate Dynamic All-Pairs Shortest Path algorithm for Undirected, Unweighted Graphs? I am looking for an algorithm involving adding unweighted edges to an empty, undirected graph (with vertices) and then for each, updating the table of shortest paths. An example is if we have vertices 1 to 5. We can see that initially, all of the paths are infinite except for the diagonal entries, i.e.:  1 2 3 4 5 1 0 i i i i 2 i 0 i i i 3 i i 0 i i 4 i i i 0 i 5 i i i i 0  where i means "infinite". But if we add an edge (1, 2) then the 1-2nd and 2-1st entries of the table are updated:  1 2 3 4 5 1 0 1 i i i 2 1 0 i i i 3 i i 0 i i 4 i i i 0 i 5 i i i i 0  Now, if we add the edge (2, 3), then obviously the 2-3rd and 3-2nd entires are updated, but also are 1-3rd and 3-1st (which has length 2, since the path is 1->2->3 and 3->2->1):  1 2 3 4 5 1 0 1 2 i i 2 1 0 1 i i 3 2 1 0 i i 4 i i i 0 i 5 i i i i 0  Now, the more edges and vertices we add, the more complex the problem becomes, since we need to update other entries in the matrix if they are connected. The one fact that I know about this kind of graph is that it is symmetric along the diagonal. I've been looking at this problem for a few hours and I do not think there is a generic algorithm that will solve the problem for a general unweighted, undirected graph. Are there resources on the problem? TheoryOverflow References on formalization What are good references in the field of formalization, as applied to mathematics and computer science? I've found [1] to be quite complete, covering topics such as modal logic and some applications. Are there other references (not only books) following this line? [1] Huth & Ryan, "Logic in Computer Science: Modelling and Reasoning about Systems" CompsciOverflow Difference between hypervisor and exokernel An hypervisor (1st type) is a software that creates and run virtual machines, managing guest's operative systems's requests to the hardware. An exokernel is an operative system kernel, that let's programs access directly to the hardware or, with the support of specific libraries that implements abstactions, run different types of executables for that architecture. Both the two are an abstraction layer between hardware and application software. Both the two let you run on the same machine different executables/software/operative systems/whatever So what's the difference between the two? StackOverflow Programmatically obtain ephemeral port with Akka If I configure an ActorSystem in Akka to use an emphemeral tcp port (more specifically, I configure the http port to 0), is there a way to programmatically obtain this port once the ActorSystem is started? Any attempts to create an actor using actorOf, and then printing out the actor path shows a locally-referenced actor. Additionally, I tried registering a RemoteLifeCycleEvent listener on the event stream, but this can only be done after the server has been started, and hence misses RemoteServerStarted event. Spring Data Neo4j in Play 2.x app using Scala I am currently working on a web service backed by Neo4j (v2.1.2, hosted on GrapheneDB) and Play (v2.3) using Scala (2.11.1) (hosted on Heroku). Until now AnormCypher connects the Play app with the Neo4J instance and all things are working as expected. However using AnormCypher doesn't feel quite right. I have to write a lot boiler plate code to convert the results I get from AnormCypher to the Scala classes which are used for the business logic. I would prefer using something similar to Spring Data Neo4j. The code in the examples looks very clean to me and I want to try it in project. All these technologies are new to me so I need some help: • How can I use Spring Data Neo4j in the play app (build.sbt, plugin.sbt, ...)? • Which difficulties do I have to expect regarding the fact that I am not using the whole Spring stack? • Which drawbacks does it have to integrate a Java based framework in a Scala app ? /r/netsec cPanel CSRF vulnerability I discovered StackOverflow How to display classpath used for "sbt run"? How can the classpath used for run task be displayed under SBT 0.13? I have found some info here https://groups.google.com/forum/#!msg/simple-build-tool/0rhVRPnjyZU/DOYAd14gh1wJ: I was dumping my classpaths as a way to troubleshoot my build recently and maybe this task can help you too: lazy val printClasspath = task { this.runClasspath.getPaths.foreach(println); None } runClasspath is a PathFinder [1] instance and you can interrogate it even further. I guess building a ':'-separated list of those paths will be easy. I don't understand where and how I should use this tip. Please advise. TheoryOverflow Perfect hashing family variation - injectivity on r disjoint sets We denote by [t] the set \{1,2,\ldots,t\}. A (n,k)-perfect hashing family is a set of functions H=\{h_i:[n]\to[k]\} such that for every set S\subset [n], |S|\leq k, there exists some h_S \in H such that h_S is injective on S. There are known (deterministic) builds of such family of size O(e^{k+log^2k}log n), while there is a \Omega(e^k \cdot \frac{log n}{\sqrt k}) lower bound , which is almost tight. We define an extension of such families as follows: A family of functions \mathcal F=\{f_i:[n]\to [k]\} is a (n,r,k)-perfect hashing family if for every set of sets, \mathcal S=\{S_1,S_2,\ldots,S_r\subseteq [n]\}, such that$$|\mathcal S|=r,\ \forall i:|S_i|=k,\ \forall j\neq i:S_i\cap S_j = \emptyset$$There exists a function f\in \mathcal F which is injective on all S_i\in \mathcal S. That is, every set of r disjoint sets of indices of size k there exists a single function which is injective on all of them. What is the smallest (n,r,k)-perfect hashing family we can build? Specifially, I'm mainly interested in the r=2 case, but this might be useful to others for general r values. QuantOverflow Historic Value at Risk - Ratios vs. Differences Quick Summary on Historic VaR Let S_0,...,S_n be the daily values of some stock (where S_0 is the current value). Then for i=1,\ldots,n we let$$\hat r_i:=S_{i-1}/S_i \quad \text{and}\quad \hat S_i := S_0\cdot \hat r_i$$Now we can estimate e.g. the 95% 1-day-VaR by looking at the (0.95n)-th smallest number amongst all the scenarios \hat S_1,...,\hat S_n and then subtracting S_0. The above approach works fine when we look at stocks since S_i>0 for all i. But what if we consider an interest rate that is potentially close to zero, and even worse, may go negative (which is the case for the German short-term treasury bills at the moment), then we have that the \hat r_i become very large and potentially negative, which renders the scenarios \hat S_i completely useless. Questions One solution might be to look at the differences, i.e. \overline r_i := S_{i-1}-S_i and \overline S_i = S_0+\overline r_i. But this completely ignores the order of magnitude of an asset. So my questions are: 1) Does anybody have an idea as to what approach is usually used in practice 2) Are differences a sensible approach at all? 3) Are there potentially other methods to avoid this problem? StackOverflow Examples of C# 5.0 async/await vs. Akka actors vs. basic fork/join? Both C# and Scala have adopted frameworks for simplifying doing asynchronous/parallel computation, but in different ways. The latest C# (5.0, still in beta) has decided on an async/await framework (using continuation-passing under the hood, but in an easier-to-use way), while Scala instead uses the concept of "actors", and has recently taken the actors implementation in Akka and incorporated it into the base library. Here's a task to consider: We receive a series of requests to do various operations -- e.g. from user input, requests to a server, etc. Some operations are fast, but some take awhile. For the slow ones, we'd like to asynchronously do the operation (in another thread) and process it when the thread is done, while still being free to process new requests. A simple synchronous loop might be (pseudo-code): while (1) { val request = waitForAnything(user_request, server_request) val result = do_request(request) if (result needs to be sent back) send_back_result(result) }  In a basic fork/join framework, you might do something like this (pseudo-code): val threads: Set[Thread] while (1) { val request = waitForAnything(user_request, server_request, termination of thread) if (request is thread_terminate) { threads.delete(request.terminated_thread) val result = request.thread_result if (result needs to be sent back) send_back_result(result) } else if (request is slow) { val thread = new Thread(() => do_request(request)) Threads.add(thread) thread.start() } else { val result = do_request(request) if (result needs to be sent back) send_back_result(result) } }  How would this look expressed using async/await and using actors, and more generally what are the advantages/disadvantages of these approach? CompsciOverflow Explanation of Tag, Index, and Offset in Direct Mapping Cache I'm going through an exercise trying to store address references into a direct mapped cache with 128 blocks and a block size of 32 bytes. The address are 20000, 20004, 20008, and 20016 in base 10. When I start to store them, I realize that the offset is the only value that changes. All of the examples I find determine hits and misses based on the target and index values, without mention of offset. What is the purpose of the offset? If the tag and index are in the cache, is it a hit regardless of the offset? Or is the offset also looked at? Lobsters Riak 2.0.0 RC1 StackOverflow Creating a tree from a text file in json - Clojure I have a text file consisting of a json value in each line. My file is as follows: {"id":"a","family":"root","parent":nil,"value":"valueofa"} {"id":"b1","family":"b","parent":"a","value":"valueofb1"} {"id":"c1","family":"c","parent":"b1","value":"valueofc1"} {"id":"c2","family":"c","parent":"b1","value":"valueofc2"} {"id":"b2","family":"b","parent":"root","value":"valueofb2"} {"id":"d1","family":"d","parent":"b1","value":"valueofd1"}  In the json given above, we the family attribute indicates a hierarchy, we would have "root" as the root,"b" as child of the "root","c" as child of "b" and and "d" would be child of "b" as well. The idea is to iterate through the file and add the node being read at its proper place in a tree. One way would be to read these entries into a "map" and then use this map for any tree-manipulations. For any complex tree manipulations, I am not sure how this would work. It is quite possible that based on a certain requirement, I might have to detach a child from an existing parent and attach it to another parent. Apparently Clojure zippers are supposed to help in this regard but I am getting a little confused by how the hierarchy of nodes work in zippers. It will be great if somebody could point me in the right direction here. How can I get Clojure :pre & :post to report their failing value? (defn string-to-string [s1] {:pre [(string? s1)] :post [(string? %)]} s1)  I like :pre and :post conditions, they allow me to figure out when I have put "square pegs in round holes" more quickly. Perhaps it is wrong, but I like using them as a sort of poor mans type checker. This isn't philosophy though, this is a simple question. It seems in the above code that I should easily be able to determine that s1 is a function argument in the :pre condition. Similarily, % in the :post condition is always the function return value. What I would like is to print the value of s1 or % when either of these respective conditions fail within the AssertionError. So I get something like (string-to-string 23) AssertionError Assert failed: (string? s1) (pr-str s1) => 23  With the AssertionError containing a single line for every variable that was identified as being from the function argument list and that was referenced in the failing test. I would also like something similar when the return value of the function fails the :post condition. This would make it trivial to quickly spot how I misused a function when trying to diagnose from the AssertionError. It would at least let me know if the value is nil or an actual value (which is the most common error I make). I have some ideas that this could be done with a macro, but I was wondering if there was any safe and global way to basically just redefine what (defn and (fn and friends do so that :pre and :post would also print the value(s) that lead to the test failing. /r/netsec I wrote this paper on WEP as an undergrad a few years ago. Would like others to point out flaws. StackOverflow What are the differences between Scala middleware choices? Note: Unfortunately this question was closed, but I'm trying to maintain it for if someone else comes along with the same question. I've been looking for a good solution to developing a service in Scala that will sit between mobile devices and existing web services. The current list of viable options are: There are probably more options out there. How does one decide which one to use? What are the traits (excuse the pun ;-) of a good Scala middleware choice. On the one side, I would like to go for Akka, because it is part of the TypeSafe Scala stack, but on the other, something like Finagle has a rich set of libraries and makes plumbing so easy. Spray looks nice and simple to use. Any advice, insights or experience would be greatly appreciated. I'm sure someone out there must have some experience with some of these that they won't mind sharing. UPDATE: I would love for this question to be reopened. A good answer to this question will help new Scalateers to avoid related pitfalls. UPDATE 2: These are my own experiences since asking this question: Finagle - I used Finagle for a project and it's rock solid. Spray - In my latest project I'm using Spray and I'm extremely happy. The latest releases are built on Akka 2 and you can run it directly with the Spray-can library which removes the need for a web server. Spray is a set of libraries, rather than a framework and is very modular. The Learn about Spray: REST on Akka video gives a great overview, and this blog at Cakesolutions shows a really nice development approach and architecture. UPDATE 3: Life moves pretty fast. If you don't stop and look around once in a while, you could miss it. - Ferris Bueller These days the choice has become simpler. In my humble opinion Spray has won the battle. It is being integrated into Akka to become the next Akka HTTP. I have been using Spray now on multiple projects and can honestly say that it's fantastic and best supported software I have ever encountered. This does not answer the initial question, but at least gives some indication on why Spray seems like the best choice in most cases. It is extremely flexible, non-blocking and very stable. It has both client-side and server-side libraries and a great testkit. Also, have a look at these stats to get an idea on performance: Web Framework Benchmarks QuantOverflow How to get Geometric Brownian Motion's closed-form solution in Black-Scholes model? The Black Scholes model assumes the following dynamics for the underlying, well known as the Geometric Brownian Motion:$$dS_t=S_t(\mu dt+\sigma dW_t)$$Then the solution is given:$$S_t=S_0\,e^{\left(\mu-\frac{\sigma^2}{2}\right)t+\sigma W_t}$$It can be shown by Ito Lemma on function f(t,W_t)=\ln S_t that this solution is correct as it leads to above dynamics. But how do we solve the above SDE originally to find this solution? Guessing the above solution to apply Ito seems unlikely to me. Halfbakery Pirate Suitability Exam (1.0) Portland Pattern Repository Category Web Gui Discomfort (by 146.233.0.201 24 hours ago) Are Desktop Computers Obsolete (by 79-64-37-209.host.pobb.as13285.net 5 hours ago) StackOverflow Scala class with covariance in derivable type of parameter I want this typeclass: abstract class Model[U](val query: TableQuery[ModelTable[U]]) { // ... }  But with covariance in the type of query. The idea is that I have functions that take an instance of U, and need access to an instance of a corresponding TableQuery[ModelTable[U]]. Adding a type parameter is nasty, because I can no longer write functions using context bounds: def f[U : Mode](u: U) = // ...  and writing out the implicit parameters everytime is quite hassle. And as for now I cannot see why this would be impossible. Given a concrete type for U, the type constraint for query should be trivial to derive for the compiler. Maybe more concrete: why can this not compile: // error: Unbound Wildcard Type abstract class Model[U](val query: _ <: TableQuery[ModelTable[U]]) { // ... }  Changing multiple sbt settings in a given scope in build.sbt I want to group settings by the context in which they apply in my build.sbt. Instead of foo in ThisBuild := bar baz in ThisBuild := bar bar in Compile := 5 quux in Compile := 7  I'd like something that roughly looks like (don't care much about the specific syntax/API, just the conceptual grouping and lack of repetition) in(ThisBuild) { foo := bar baz := bar } in(Compile) { bar := 5 quux := 7 }  Is such a thing possible/clean/idiomatic? Lobsters Your Voice Assistant is Mine: How to Abuse Speakers to Steal Information and Control Your Phone StackOverflow Preferred way to create a Scala list There are several ways to construct an immutable list in Scala (see contrived example code below). You can use a mutable ListBuffer, create a var list and modify it, use a tail recursive method, and probably others that I don't know about. Instinctively, I use the ListBuffer, but I don't have a good reason for doing so. Is there a preferred or idiomatic method for creating a list, or are there situations that are best for one method over another? import scala.collection.mutable.ListBuffer // THESE are all the same as: 0 to 3 toList. def listTestA() ={ var list:List[Int] = Nil for(i <- 0 to 3) list = list ::: List(i) list } def listTestB() ={ val list = new ListBuffer[Int]() for (i <- 0 to 3) list += i list.toList } def listTestC() ={ def _add(l:List[Int], i:Int):List[Int] = i match { case 3 => l ::: List(3) case _ => _add(l ::: List(i), i +1) } _add(Nil, 0) }  Lobsters The Great Whitespace Debate I filed this as ‘rant’ and ‘satire’ because it’s not serious, but I absolutely love the writing here. Comments StackOverflow Lein test failing with No such var: leiningen.util.injected/add-hook I get this error in a new project, without touching tests or test related configuration Exception in thread "main" java.lang.RuntimeException: No such var: leiningen.util.injected/add-hook, compiling:(NO_SOURCE_PATH:1) at clojure.lang.Compiler.analyze(Compiler.java:6235) at clojure.lang.Compiler.analyze(Compiler.java:6177) at clojure.lang.CompilerInvokeExpr.parse(Compiler.java:3452) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6411) at clojure.lang.Compiler.analyze(Compiler.java:6216) at clojure.lang.Compiler.analyze(Compiler.java:6177) at clojure.lang.CompilerBodyExprParser.parse(Compiler.java:5572) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6409) at clojure.lang.Compiler.analyze(Compiler.java:6216) at clojure.lang.Compiler.analyze(Compiler.java:6177) at clojure.lang.CompilerIfExprParser.parse(Compiler.java:2597) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6409) at clojure.lang.Compiler.analyze(Compiler.java:6216) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6397) at clojure.lang.Compiler.analyze(Compiler.java:6216) at clojure.lang.Compiler.analyze(Compiler.java:6177) at clojure.lang.CompilerBodyExprParser.parse(Compiler.java:5572) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6409) at clojure.lang.Compiler.analyze(Compiler.java:6216) at clojure.lang.Compiler.analyze(Compiler.java:6177) at clojure.lang.CompilerBodyExprParser.parse(Compiler.java:5572) at clojure.lang.CompilerTryExprParser.parse(Compiler.java:2091) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6409) at clojure.lang.Compiler.analyze(Compiler.java:6216) at clojure.lang.Compiler.analyze(Compiler.java:6177) at clojure.lang.CompilerBodyExprParser.parse(Compiler.java:5572) at clojure.lang.CompilerFnMethod.parse(Compiler.java:5008) at clojure.lang.CompilerFnExpr.parse(Compiler.java:3629) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6407) at clojure.lang.Compiler.analyze(Compiler.java:6216) at clojure.lang.Compiler.eval(Compiler.java:6462) at clojure.lang.Compiler.eval(Compiler.java:6455) at clojure.lang.Compiler.eval(Compiler.java:6431) at clojure.coreeval.invoke(core.clj:2795) at clojure.maineval_opt.invoke(main.clj:296) at clojure.maininitialize.invoke(main.clj:315) at clojure.mainnull_opt.invoke(main.clj:348) at clojure.mainmain.doInvoke(main.clj:426) at clojure.lang.RestFn.invoke(RestFn.java:421) at clojure.lang.Var.invoke(Var.java:405) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.Var.applyTo(Var.java:518) at clojure.main.main(main.java:37) Caused by: java.lang.RuntimeException: No such var: leiningen.util.injected/add-hook at clojure.lang.Util.runtimeException(Util.java:156) at clojure.lang.Compiler.resolveIn(Compiler.java:6694) at clojure.lang.Compiler.resolve(Compiler.java:6664) at clojure.lang.Compiler.analyzeSymbol(Compiler.java:6625) at clojure.lang.Compiler.analyze(Compiler.java:6198) ... 42 more Lobsters Compiling Rust for the Rasberry Pi /r/compsci Advice for Real World Application of Encryption Strategy? Person A has a small piece of data they want to keep secure, even from themselves for 3 months. Person B wants to ensure person A can secure their data for that amount of time. Person C wants person A's data. Person C is not above stealing from person A if they can. Person A trusts person B but knows person B could be compromised by person C. __ My initial attempt to solve this was by suggesting split key encryption: The data is encrypted and A and B get a key (using something like Cloudflare's Red October). B won't release his key to A until 3 months have passed. Downsides are though: What if B dies? Because C can compromise B, if C steal's A's key it could all be over. __ So I'm trying to find a better plan, any suggestions? submitted by pikadrew [link] [2 comments] Halfbakery Low Rent 23/7 Internet Cafe (0.0) Portland Pattern Repository Stable Abstractions Principle (by 79-64-37-209.host.pobb.as13285.net 25 hours ago) Eighty Twenty Rule (by 146.233.0.202 25 hours ago) DataTau A/B testing - statistical hypothesis testing vs multi-armed bandit /r/clojure Mobile App Development with ClojureScript and Cordova TheoryOverflow Video lectures on type systems [migrated] For my job, I need to pick up a working understanding of the implementation of type systems (in particular, how to write typing rules based on a design document). I've been given a copy of Types and Programming Languages and that's great, but I'd really like to watch someone work through it. In my school, the course that teaches this is called "Programming Languages," but searching for "video lectures programming languages" turns up lots of stuff about how to learn C++, which I'm not interested in. I couldn't find anything on MIT Open Courseware. I'd really like video lectures that go through TAPL, but I'll settle for anything useful. Are there good resources out there? /r/compsci Biological Realms in Computer Science StackOverflow Compare two Maps in Scala Is there any pre-defined function that I can use to compare two Maps based on the key and give me the difference? Right now, I iterate Map1 and foreach key, I check if there is an element in Map2 and I pattern match to find the difference. Is there a much elegant way to do this? /r/netsec Hemlis is in closed Beta and has a new video demo. /r/osdev OS Resource center StackOverflow Spark Tachyon: How to delete a file? In Scala, as an experiment I create a sequence file on Tachyon using Spark and read it back in. I want to delete the file from Tachyon using the Spark script also. val rdd = sc.parallelize(Array(("a",2), ("b",3), ("c",1))) rdd.saveAsSequenceFile("tachyon://127.0.0.1:19998/files/123.sf2") val rdd2 = sc.sequenceFile[String,Int]("tachyon://127.0.0.1:19998/files/123.sf2")  I don't understand the Scala language very well and I cannot find a reference about file path manipulation. I did find a way of somehow using Java in Scala to do this, but I cannot get it to work using Tachyon. import java.io._ new File("tachyon://127.0.0.1:19998/files/123.sf2").delete()  Why is the _0 Nat in Shapeless a class instead of an object? I'm trying to understand Shapeless, and I came across this: /** * Base trait for type level natural numbers. * * @author Miles Sabin */ trait Nat { type N <: Nat } /** * Encoding of successor. * * @author Miles Sabin */ case class Succ[P <: Nat]() extends Nat { type N = Succ[P] } /** * Encoding of zero. * * @author Miles Sabin */ class _0 extends Nat { type N = _0 }  _0 is a special and unique case, like Nil for a List. _0 has no predecessor. Why isn't it an object/case object (which is a singleton)? HLists seem to do this. /** * HList ADT base trait. * * @author Miles Sabin */ sealed trait HList /** * Non-empty HList element type. * * @author Miles Sabin */ final case class ::[+H, +T <: HList](head : H, tail : T) extends HList { override def toString = head+" :: "+tail.toString } /** * Empty HList element type. * * @author Miles Sabin */ sealed trait HNil extends HList { def ::[H](h : H) = shapeless.::(h, this) override def toString = "HNil" } /** * Empty HList value. * * @author Miles Sabin */ case object HNil extends HNil  Halfbakery Touchscreen Gamepad (0.0) Sewing Cafe (2.0) DataTau 9 Visualisation Blogs Followed at Boxever /r/compsci CS major and Astronomy minor. Is it a good combo? submitted by h-b [link] [26 comments] StackOverflow Get only the Header using the WS library from play framework 2.2.1. (Scala) I am stuck using the 2.2.1 version of the play framework and I am trying to check the Content-Length of a request for a file using the WS libraries without actually downloading the body of the request. In the version 2.3.x of play there has been a new method added getStream() that lets you process the request yourself with a stream, but I am stuck on version 2.2.1 for now and cannot find a similar way. So far I have tried this but it never seems to get a response: import play.api.libs.ws.{ResponseHeaders, WS} import play.api.libs.iteratee.Concurrent.joined WS.url(url).get { (rsp: ResponseHeaders) => val (wsConsumer, _) = joined[Array[Byte]] val size = rsp.headers.get("Content-Length").map(_.head).fold(0L)(_.toLong) play.Logger.info(s"Size: size") wsConsumer }  Having trouble with implicit conversion in scala Having this code case class Workspace(ident: Long, name: String) case class Project(ident: Long, name: String) implicit def workspaceJSON: JSONR[Workspace] = new JSONR[Workspace] { def read(json: JValue) = Workspace.applyJSON(field[Long]("id"), field[String]("name"))(json) } implicit def projectJSON: JSONR[Project] = new JSONR[Project] { def read(json: JValue) = Project.applyJSON(field[Long]("id"), field[String]("name"))(json) } def parseEnt[T: JSONR](json: JValue): Either[String, T] = fromJSON[T](json).toEither.left.map{ _.toString } def fetchProjects(ws: Workspace): Either[String, Project] = { parseEnt(parse("some text")) }  Which fails to compile on parseEnt(parse("some text")) with ambiguous implicit values: both method taskJSON in class Fetcher of type => Fetcher.this.JSONR[types.Task] and method workspaceJSON in class Fetcher of type => Fetcher.this.JSONR[Fetcher.this.Workspace] match expected type Fetcher.this.JSONR[T]  Is there a way to assure scala, that in this case I want type variable T to be a Project and choose projectJSON function to parse it? Or if I'm doing it wrong, then how do it in right way? presentation compiler: type completion in method call with more than one argument I load source into compiler with askReload, and then i try to call askTypeCompletion after . (dot). Completion in first case (method with 2 arguments) is not working, but completion in method call with one arg works as expected.  val list = Seq(1,2) def add(x: Int, y: Int): Int = x + y def minusOne(x: Int) = x - 1 add(list.<completion is not working) minusOne(list.<works fine>)  what's interesting is if i have code:  implicit class OptionW[T](opt: Option[T]) { def cata[A](some: T => A, none: A) = opt.map(some) getOrElse none } Option("").cata(x => x.<not working>)  completion after dot is not working again, but if i type comma after dot and then try again to complete after dot, it works: Option("").cata(x => x.<works!>,) Is it some bug or expected behaviour? Spark - scala: shuffle RDD / split RDD into two random parts randomly How can I take a rdd array of spark, and split it into two rdds randomly so each rdd will include some part of data (lets say 97% and 3%). I thought to shuffle the list and then shuffledList.take((0.97*rddList.count).toInt) But how can I Shuffle the rdd? Or is there a better way to split the list? /r/compsci Level-Up Your Machine Learning StackOverflow How to append or prepend an element to a tuple in Scala I have a tuple and want to add an element without loosing type safety. This is what I want to achieve: val tuple = ("", 1, 1f) // (String, Int, Float) val newTuple:(String, Int, Float, Double) = tuple :+ 1d  Lobsters Flock: Private Contact and Calendar Cloud Sync for Android /r/compsci Masters in Computer Science, non-computer undergrad degree (Criminal Justice) I recently graduated with a 3.92 in criminal justice (BS). I am considering looking into a masters in computer science program since honestly my degree is fairly useless on its own. I know some schools offer bridge programs and such (I'm in the NY-NJ area) but don't really know where to begin or if this is even a good idea. Anyone have any advice? My reason for pursuing a computer science degree is that I have an interest in programming (have only recently started on my own) and have a love for computers in general. I also think that it will greatly help my career and life and doing this now while I'm young will help me greatly in the future. Thank you. submitted by pattywagon [link] [2 comments] /r/clojure CLJS Fiddle wins the Lisp in Summer Projects competition Clojure implementation of the propagator computational model CompsciOverflow Computer resource under-usage [on hold] here is a list of the core components of my Dell computer: Processor: dual core Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz RAM: 96 GB System: Windows 7 64-bit Hard drives: 2TB SATA Video: NVIDIA Quadro 600 I'm trying to make the most use out of these resources. I'm running a few high intensity applications and processes right now, but I see in my Task Manager that the system is only using around 10% (or less) of both the CPU and RAM. (See screenshot) Why is this? Can somebody help me understand where the bottleneck is on my system? Thanks Conor /r/netsec Stealthy ransomware 'Critroni' uses Tor, could replace Cryptolocker StackOverflow Is it possible to have two or more context bound classes in Scala How do I specify that I need e.g. a ClassTag AND an Ordering for T? Example  def sort[T: ClassTag <and> Ordering](future: Future[Seq[T]]): Future[Seq[T]]  /r/netsec Security researcher Jonathan Zdziarski reveals Apple’s complex tools inside iOS that would allow Apple to share certain user data at the request of law enforcement without the user knowledge. StackOverflow Reading web page with log in (clojure) I need to download some contents of a web page daily and I plan on using enlive for that. Trouble is, that I need to log in in with a POST first and the authentification of the page that I am interested in is then done with the session's cookies. So I can't just use (html/html-resource (java.net.URL. url))  I didn't find a way to do this in clojure. Otherwise doing the reading in Java would be fine as well. In the end it should work as a worker on heroku. Thanks! Lobsters What are you working on this week? It’s Monday, so it is time for our weekly “What are you working on?” thread. Please share links and tell us about your current project. Do you need feedback, proofreading, collaborators? TheoryOverflow Deterministic Buchi + its complement covers LTL? It is well known that deterministic Buchi automata (DBA) are less expressive than non-deterministic Buchi automata (NBA), and in particular DBA are not enough to cover linear temporal logic (LTL). However, the argument often used is that DBA are not closed under complementation - that is there is a language \mathcal D recognized by DBA, but \mathcal D^c is not recognized by any DBA. As an example, \mathcal D = (b^*a)^\omega and \mathcal D^c = (a+b)^*a^\omega. I wonder though, whether for any \omega-regular language \mathcal L it holds either \mathcal L is recognized by some DBA, or \mathcal L^c is recognized by some DBA. If not, what is an example of a language which is not recognizable by DBA and which complement is not recognizable either? Lobsters StackOverflow Update: 560M Pageviews a Month, 25 Servers Ham Transmitter Fingerprinting Undeadly g2k14: Ken Westerback on DHCP and dump(8) Having missed Ljubljana 1, I looked forward to Ljubljana 2 with great expectations. I was not disappointed! Mitja ran a great hackathon with a nice site and an excellent city around it. Read more... StackOverflow Add plugins under a same project in sbt I'm trying to build a Scala project aggregated by multiple projects, one of which is an sbt plugin. I would like to use this plugin in another subproject under the same parent project, but I don't quite understand how to do this. My "build.sbt" in the project root is like this: lazy val plugin = project .in(file("sbt-Something")) .dependsOn(lib) .settings(common: _*) .settings(name := "My plugin", sbtPlugin := true) lazy val examples = project .in(file("examples")) .dependsOn(lib, plugin) .settings(common: _*) .settings(name := "Examples")  How to add the plugin as a plugin to project examples? Why does Clojure's for macro only accept one body expression? Clojure's for macro accepts two arguments: a sequence of binding forms, and a body expression. As such, if I want to do multiple things in the loop, I have to wrap multiple expressions in a do block to make them a single expression. Compare: (doseq [a (range 3)] (prn 'a a) (inc a))  to: (for [a (range 3)] (prn 'a a) (inc a))  The doseq works as expected. The for complains: clojure.lang.ArityException: Wrong number of args (3) passed to: core/for  My question is, why doesn't this work? Why didn't the designer(s) of Clojure allow multiple "body" expressions in the for loop, like they did in doseq and when? It's not as if there is any semantic ambiguity, right? QuantOverflow plotting High Frequency finance data using quantmod [on hold] Hi I have data in XTS format like this  > ITXxts Time Price Volume 2014-07-18 09:00:00 67.63 460 2012-04-27 09:00:00 67.63 73 2012-04-27 09:00:00 67.63 85 2012-04-27 09:00:01 67.63 3 2012-04-27 09:01:03 67.79 1 2012-04-27 09:01:16 67.64 91 2012-04-27 09:05:50 67.83 75  Now aggregated the series into 5 minute blocks tsagg5min =aggregatets(ITXxts,on="minutes",k=5) 2014-07-18 09:05:00 67.45 43 2014-07-18 09:10:00 67.82 12 2014-07-18 09:15:00 67.91 341 2014-07-18 09:20:00 67.70 275  chartSeries(tsagg5min) Now my question is how can I plot it in the OHLC format in such a way that It calculates the the Open High Low and Close of each 5 minute chunk in which the data was aggregated TheoryOverflow Lower bounds for Polynomials computing the boolean functions Expressing a boolean function f :\{ 0,1 \}^{n} \rightarrow \{0,1 \} using a polynomial P(x_{1},...,x_{n}), where x_{1},...,x_{n} may be integer, finite fields, or other fields. One of the most popular examples is Fourier Analysis e.g. http://analysisofbooleanfunctions.org/ . Another topic is transforming small depth circuits to polynomials over integer with small degree and magnitudes of small weights. For example, transformation from ACC circuits to polynomials proved by Yao and Beigel and Tarui is notable. My questions are: Qestion A. Is there a list of results about lower bounds and remarkable techniques for polynomials computing boolean functions. I know the Williams's separation between NEXP and ACC. Question B. Does the recent separation result about ACC \circ THR circuits by Williams contain some interpretation relating with polynomials computing boolean funcitons ? Question C. Is proving lowerbounds for the number of monomials or degrees or other formally defined compleixty measures relating wtih the algebraic compleixty theory and arithmetic circuit complexity established by Valiant with VP v. VNP problem?? I am reading the survey written by Shiplka, and have met a defference between boolean compleixty and arithmetic compleixty: semantic versus syntatic. Here is a response to the J's question. The survey has a remark about defference between polynomial and polynomial function. Taking an example, x^{2}-x is a polynomial which is NOT identicdally zero but is a polynomial function GF(2)^{n}\rightarrow GF(2) identically zero over GF(2). In algebraic compleixty, Valiant's question is about compleixyt of expressioning polynomial (not polynomial funciton). However, I mainly want to consider polynomial funcitons. Thanks. QuantOverflow How to price a Swing Option? I'm working in the commodity market and I've to price Swing Options with MATLAB, preferably with finite element. Has anyone already priced these kind of derivatives? I'm thinking about using the structure for the pricing of an American Option and then do it iteratively. More details about Swing Options are included in this paper. Note that swing options are really useful in commodity markets because you can exercise them more than once (like American options); obviously there are some constraints that limit you. I've already tried to price them with Least Squares Monte Carlo method (using the algorithm presented by Longstaff and Schwartz). Now I want to price them with finite element but I'm having some difficulties. In particular I'm pricing them without jumps, so I'm using an EDP discretized (and not a PIDE). I'd like to know if anyone already implemented such a thing? StackOverflow scala boolean variables need to change dynamicly I have this method that I need to evaluate an expression based on these Boolean values: var Ma, Co, Ar, Le, St, Ri: Boolean = false def Logic: Boolean = (Ma & (!Ma | Co) & (!Ma | Ar) & (!Co | Ma) & (!Ar | Ma) & (!Ar | Le | St | Ri))  In main() method I have: def main(args: Array[String]) { val ls: List[String] = List("Ma", "Co", "Ar") //now I need to change the value of Ma, Co and Ar variables to "true"?? }  Is there a general way that may help to change the value of only these Boolean variables to true that are found in this list? How to pattern-match an Option in the underlying? It says in Scala, pattern-matching will be converted to an invocation of unapply(...):Option[...]. Here I defined an object with unapply: object Hello { def unapply(s:String): Option[String] = Some(s) }  Then I can use it in pattern-matching: "hello" match { case Hello(s) => println(s) }  Actually(maybe) it will be converted to: Hello.unapply("hello") match { case Some(s) => println(s) case _ => }  But the Some here is also a case class, so the Some.unapply will be called. That should be infinite. Which is not possible. So how do Scala pattern matching the Option in the underlying? Surprising behaviour related to records, protocols and compilation I encountered a, to me, slightly surprising behaviour seemingly related to clojure records. The setup is as follows: 1. One namespace defines a record type: (ns defrecordissue.arecord) (defrecord ARecord [])  2. Another namespace defines a protocol, and extends it to the record type defined in 1: (ns defrecordissue.aprotocol (:require [defrecordissue.arecord]) (:import [defrecordissue.arecord ARecord])) (defprotocol AProtocol (afn [this])) (extend-protocol AProtocol ARecord (afn [this] 42))  3. A third namespace constructs an instance of the record and invokes the protocol function on the record. (ns defrecordissue.aot1 (:require [defrecordissue.aprotocol] [defrecordissue.arecord])) (defrecordissue.aprotocol/afn (defrecordissue.arecord/->ARecord))  When the defrecordissue.aot1 namespace is compiled, in my case using lein compile defrecordissue.aot1, compilation fails with the following exception: Exception in thread "main" java.lang.IllegalArgumentException: No implementation of method: :afn of protocol: #'defrecordissue.aprotocol/AProtocol found for class: defrecordissue.arecord.ARecord, compiling:(aot1.clj:5:1) at clojure.lang.CompilerInvokeExpr.eval(Compiler.java:3463) at clojure.lang.Compiler.compile1(Compiler.java:7153) at clojure.lang.Compiler.compile(Compiler.java:7219) at clojure.lang.RT.compile(RT.java:398) at clojure.lang.RT.load(RT.java:438) at clojure.lang.RT.load(RT.java:411) at clojure.coreloadfn__5018.invoke(core.clj:5530) at clojure.coreload.doInvoke(core.clj:5529) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.coreload_one.invoke(core.clj:5336) at clojure.corecompilefn__5023.invoke(core.clj:5541) at clojure.corecompile.invoke(core.clj:5540) at usereval7.invoke(NO_SOURCE_FILE:1) at clojure.lang.Compiler.eval(Compiler.java:6619) at clojure.lang.Compiler.eval(Compiler.java:6609) at clojure.lang.Compiler.eval(Compiler.java:6582) at clojure.coreeval.invoke(core.clj:2852) at clojure.maineval_opt.invoke(main.clj:308) at clojure.maininitialize.invoke(main.clj:327) at clojure.mainnull_opt.invoke(main.clj:362) at clojure.mainmain.doInvoke(main.clj:440) at clojure.lang.RestFn.invoke(RestFn.java:421) at clojure.lang.Var.invoke(Var.java:419) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.Var.applyTo(Var.java:532) at clojure.main.main(main.java:37) Caused by: java.lang.IllegalArgumentException: No implementation of method: :afn of protocol: #'defrecordissue.aprotocol/AProtocol found for class: defrecordissue.arecord.ARecord at clojure.core_cache_protocol_fn.invoke(core_deftype.clj:541) at defrecordissue.aprotocolfn__40G__35__45.invoke(aprotocol.clj:5) at clojure.lang.AFn.applyToHelper(AFn.java:161) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.lang.CompilerInvokeExpr.eval(Compiler.java:3458) ... 25 more  If I change 3) to construct the record class directly, like so: (ns defrecordissue.aot2 (:require [defrecordissue.aprotocol] [defrecordissue.arecord])) (defrecordissue.aprotocol/afn (defrecordissue.arecord.ARecord.))  Compilation succeeds. My suspicion is that this is somehow related to http://dev.clojure.org/jira/browse/CLJ-371, but I don't understand exactly what is happening. I should also add that without the lein clean, compilation succeeds the second time, since a class for the record is now available on the classpath. Therefore, I can get around this problem by AOT-compiling the namespace defining the record type. I created a simple leiningen project on GitHub that illustrates the issue, see README for usage: https://github.com/ragnard/defrecordissue Why am I seeing this behaviour, and what is the correct way to avoid it? UPDATE I added a new branch to the GitHub repo better illustrating the core issue: https://github.com/ragnard/defrecordissue/tree/more-realistic/ The problem occurs regardless of where (ie. in which namespace) the record instance is constructed. Encoding singleton objects as lazy vals Context We are working on a static verifier for Scala programs (early work described in this Master's thesis), and the current focus lies on verifying Scala features that involve lazy evaluation. We are mainly interested in the semantics (behaviour) of the features, not in other (nevertheless important) aspects such as comprehensibility or conciseness. To simplify things, we - for the time being - ignore special roles that singleton objects could have. For example, that some are companion objects (which is probably orthogonal w.r.t. to their lazy nature), or that some are package objects. Properties of lazy vals and singleton objects Lazy vals Assume a lazy val lazy val v = I  where I is the initialiser block, i.e., the code that determines the value of the lazy val. The initialiser block I is executed when the lazy val v is dereferenced for the first time. Singleton objects Assume a singleton object object Foo { C1 val v1 = I1 var v2 = I2 lazy val v3 = I3 def m() {C2} }  where C1 is the code making up the constructor of object Foo, where I1 to I3 are again initialiser blocks, and where C2 is the body of method m. When the object Foo is first used (dereferenced, or assigned to a variable/field), then C1, I1 and I2 are executed. I3 is only executed when Foo.v3 is dereferenced (since v3 is a lazy val) and C2 is executed whenever m is called. Question Consider this version of Foo, where the singleton object has been encoded by a lazy val and an anonymous class: // Should probably sit in a package object lazy val Foo = new { C1 val v1 = I1 var v2 = I2 lazy val v3 = I3 def m() {C2} }  Can anybody think of a reason for why the encoding of the singleton object Foo as a lazy val would show a different behaviour than the original singleton object? That is, are there (corner) cases where the encoded version would have a different semantics than the original code? Fefe Aktuelles Highlight aus dem Josef-S-Gerichtsverfahren: ... Aktuelles Highlight aus dem Josef-S-Gerichtsverfahren: Schaut mal, wie der fiese Gewalttäter in den Gerichtssaal geführt wurde. Damit auch wirklich keiner der Schöffen oder Beobachter den Eindruck hat, dass es sich hier nicht um einen hochkriminellen Gewalttäter handelt. Human Rights Watch kümmert sich mal darum, dass die ... Portland Pattern Repository Translator Pattern (by 79-64-37-209.host.pobb.as13285.net 7 hours ago) TheoryOverflow References to learn more about graph laplacian. I have vaguely heard of this connection between random matrix theory and graphs (the spectral gap of their laplacians) on compact Riemann surfaces. • Can someone give a pedagogic reference which helps learn this subject? • Also is there something called "free probability theory" which is somehow equivalent to random matrix theory? • Would you call the usual QFT of Hermitian matrices a random matrix theory? StackOverflow Ansible synchronize prompts passphrase even if already entered at the beginning The synchronize module of Ansible (v1.6.5) prompts for the passphrase (Enter passphrase for key) even though I already entered it at the beginning of running the playbook. Any idea why? I run my playbook with the following options: -u myuser --ask-sudo-pass --private-key=/path/to/id_rsa  Here is my synchronize task: - name: synchronize source files in src location sudo: yes synchronize: src={{local_src}} dest={{project_dirs.src}} archive=yes delete=yes rsync_opts=["--compress"] when: synchronize_src_files  UPDATE with ssh-agent Following the advice of Lekensteyn, I tried with ssh-agent. I do not have a prompt anymore but the task fails. What am I missing? eval ssh-agent -s ssh-add ~/.ssh/id_rsa  The error: TASK: [rolebooks/project | synchronize source files in src location] ********** failed: [10.0.0.101] => {"cmd": "rsync --delay-updates -FF --compress --delete-after --archive --rsh 'ssh -i /home/vagrant/.ssh/id_rsa -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" [--compress] --out-format='<<CHANGED>>%i %n%L' /projects/webapp mike@10.0.0.101:/var/local/sites/project1/src", "failed": true, "rc": 12} msg: sudo: no tty present and no askpass program specified rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.0]  CompsciOverflow difference between multilayer perceptron and linear regression What is the difference between multilayer perceptron and linear regression classifier. I am trying to learn a model with numerical attributes, and predict a numerical value. Thanks Understanding Property Testing with a toy example I am newbie with this property testing and I am trying to understand it with a few examples. I first dealt with a toy example. I did not understand the first step of the test in the following slide. Why is the first step S= 2/\epsilon? Could someone please explain me in more detail (i.e. with a concrete example)? I would appreciate with that. Lobsters Top 5 ES6 Harmony Features StackOverflow Scala - define implicitNotFound in implicit conversion I have a class Foobar[A, B](x: A) and an implicit conversion method to turn Foobar(x) => x if there is evidence that A <:< B. This currently works, so if I call Foobar(x).xyzzy where xyzzy is a method defined on x, as long as evidence exists that A <:< B the conversion works fine. My problem is that if no evidence that A <:< B exists, invoking Foobar(x).xyzzy errors with the message that xyzzy is not a member of Foobar. This is true, and I understand why this happens, but I want the error in this case to instead say Cannot prove A <:< B or something to that effect. I think implicitNotFound should help me do this, but I'm having trouble making it work. Example: import scala.language.implicitConversions class Qwerty[A](val _value: A) { def toFoobar[B](func: A => B): Foobar[A,B] = new Foobar[A,B](_value, func) } object Qwerty { implicit def wrapQwerty[A](anything: A): Qwerty[A] = new Qwerty(anything) implicit def unwrapQwerty[A](qwerty: Qwerty[A]): A = qwerty._value implicit def unwrapFoobar[A, B >: A](foobar: Foobar[A, B]): B = foobar.func(foobar._value) } class Foobar[A, B](val _value: A, val func: A => B) { } import Qwerty._ println(5.toFoobar(_ + 2) - 1) // works println(5.toFoobar(_ + "bad type") - 1) // errors (correctly) but I want error message to say "type mismatch"  puppet facter last run status Is it possible for facter to show the last status of puppet runs? typically puppet stores the result in YAML format in /var/lib/puppet/state/last_run_report.yaml any way to get these values as a facter? Best way to serialize a large amount of data using Scala 2.9 I want to serialize a Graph described as a LinkedHashMap using Scala version 2.9.3. and then deserialize/read it with another program. The amount of nodes and edges is in the range of millions and I'm planning to use Spark to parallelize the execution in the near future*. I've spent a lot of time reading about the different options but I'm still clueless about which one is the best. So, I need some advice about libraries and some links to examples. *: That's why I use Scala 2.9.3. /r/compsci The Complexity of Computing a Nash Equilibrium Lobsters Designing a system that doesn't drop messages on the floor StackOverflow Scala value toInt is not a member of Option[String] I am new with Scala and I can't figure this out. I am sure it is a simple solution but as a newbie I can't find it. I have this code val resID = CartManager.getReservation(request) val id = if (resID.isDefined && resID.get.status == "FOUND") resID.get.id.toInt else -1  and it gives me an error: value toInt is not a member of Option[String]  How can I get the id of the object Reservation if it is defined? Thank you Scala: how can I generate numbers according to an expected distribution? Sometimes Scala's random doesn't help when wanting to generate random numbers of smaller ranges or when you already know that certain numbers already have an associated probability attached to them. Obviously, scala.util.Random.nextInt won't get the whole job done. How do I select numbers according to their weights? CompsciOverflow Closed form solution for a single layer linear perceptron Let f be a one-layer neural network which is linear (ie. no activation function). Let it have p inputs and q outputs. These are fully connected by weights W. We have n inputs x \in \mathbb{R}^d and n outputs y \in \mathbb{R}^{d'}. We arrange these (column) vectors into matrices Y and X, and the weights into a matrix, so that we can express the computation of the network on the whole dataset as WX = Y (that is, if we could find the perfect W). Just to clarify, the network contains no hidden layer, just the input and output layers. Training the network is a matter of choosing W to minimize e_W = \sum_{i=1}^n||y_i - Wx_i||^2. It seems to me that this problem should have a closed form solution that can be solved by a linear solver in some way. Unfortunately, my linear algebra is lacking. Is there a closed form solution? If not, where is the difficulty? StackOverflow Difference of pattern matching for "::(a,b)" and "List(a,b)" To use :: to pattern match a list: val ::(a, tail) = List(1,2,3) // a: Int = 1 // tail: List[Int] = List(2, 3)  But with List: val List(a, tail) = List(1,2,3) // match error  I tried to figure out why the later one can't be matched, but without lucky. I see in the abstract class SeqFactory, there is a unapplySeq method: def unapplySeq[A](x: CC[A]): Some[CC[A]] = Some(x)  But when x is List[Int], the return value is still Some[List[Int]], I can't find where the magic is Fefe Schlagzeile des Tages: "Palästinenser lassen mit Sprengstoff ... Portland Pattern Repository Robinson Arithmetic (by hide.worldspan.com 29 hours ago) High Scalability StackOverflow Update: 560M Pageviews a Month, 25 Servers, and It's All About Performance The folks at Stack Overflow remain incredibly open about what they are doing and why. So it’s time for another update. What has Stack Overflow been up to? The network of sites that make up StackExchange, which includes StackOverflow, is now ranked 54th for traffic in the world; they have 110 sites and are growing at a rate of 3 or 4 a month; 4 million users; 40 million answers; and 560 million pageviews a month. This is with just 25 servers. For everything. That’s high availability, load balancing, caching, databases, searching, and utility functions. All with a relative handful of employees. Now that’s quality engineering. This update is based on The architecture of StackOverflow (video) by Marco Cecconi and What it takes to run Stack Overflow (post) by Nick Craver. In addition, I’ve merged in comments from various sources. No doubt some of the details are out of date as I meant to write this article long ago, but it should still be representative. Stack Overflow still uses Microsoft products. Microsoft infrastructure works and is cheap enough, so there’s no compelling reason to change. Yet SO is pragmatic. They use Linux where it makes sense. There’s no purity push to make everything Linux or keep everything Microsoft. That wouldn’t be efficient. Stack Overflow still uses a scale-up strategy. No clouds in site. With their SQL Servers loaded with 384 GB of RAM and 2TB of SSD, AWS would cost a fortune. The cloud would also slow them down, making it harder to optimize and troubleshoot system issues. Plus, SO doesn’t need a horizontal scaling strategy. Large peak loads, where scaling out makes sense, hasn’t been a problem because they’ve been quite successful at sizing their system correctly. So it appears Jeff Atwood’s quote: "Hardware is Cheap, Programmers are Expensive", still seems to be living lore at the company. Marco Ceccon in his talk says when talking about architecture you need to answer this question first: what kind of problem is being solved? First the easy part. What does StackExchange do? It takes topics, creates communities around them, and creates awesome question and answer sites. The second part relates to scale. As we’ll see next StackExchange is growing quite fast and handles a lot of traffic. How does it do that? Let’s take a look and see…. Stats CompsciOverflow Efficient way to check if a graph node's children were visited during traversal [on hold] I have a generic traversal algorithm for graphs: Traversal (container) { while (container is not empty ){ visit container.pop() push new nodes } }  and I want to make a time-efficient "child head count" for nodes: Traversal (container) { while (container is not empty ){ visit container.pop() push new nodes close parent nodes if last child } }  The best way that i could think of was to have pointers to "visited" booleans of the node's children and just "and" them up as such: node1 -> [node2(true), node5(false), node7(true)] node1.all_children_visited() = false Is there a way to do this with lower than O(n) complexity? /r/netsec Hardening the browser address space infra-talk jtimberman’s Code Blog 2014-07-21 15:50:56 This post will illustrate load_current_resource and a basic use of chef-shell. The chef-shell is an irb-based REPL (read-eval-print-loop). Everything I do is Ruby code, just like in Chef recipes or other cookbook components. I’m going to use a package resource example, so need privileged access (sudo). The chef-shell program loads its configuration, determines what session type, and displays a banner. In this case, we’re taking all the defaults, which means no special configuration, and a standalone session. To evaluate resources as we’d write them in a recipe, we need to switch to recipe mode. I can do anything here that I can do in a recipe. I could paste in my own recipes. Here, I’m just going to add a package resource to manage the vim package. Note that this works like the “compile” phase of a chef-client run. The resource will be added to the Chef::ResourceCollection object. We’ll look at this in a little more detail shortly. I’m done adding resources/writing code to test, so I’ll initiate a Chef run with the run_chef method (this is a special method in chef-shell). Let’s take a look at what’s happening. Note that we have INFO and DEBUG output. By default, chef-shell runs with Chef::Log#level set to :debug. In a normal Chef Client run with :info output, we see the first line, but not the others. I’ll show each line, and then explain what Chef did. There is a timestamp, the resource, package[vim], the action install Chef will take, and the location in the recipe where this was encountered. I didn’t specify one in the resource, that’s the default action for package resources. The irb#1 line 1 just means that it was the first line of the irb in recipe mode. Chef chooses the default provider for each resource based on a mapping of platforms and their versions. It uses an internal class, Chef::Version::Comparable to do this. The system I’m using is a Debian “testing” system, which has the codename jessie, but it isn’t a specific release number. Chef knows that for all debian platforms to use the apt package provider, and that’ll do here. This output is the load_current_resource method implemented in the apt package provider. The check_package_state method does all the heavy lifting. It runs apt-cache policy and parses the output looking for the version number. If we used the :update action, and the installed version wasn’t the same as the candidate version, Chef would install the candidate version. Chef resources are convergent. They only get updated if they need to be. In this case, the vim package is installed already (our implicitly specified action), so we see the following line: Nothing to do, Chef finishes its run. Modifying Existing Resources We can manipulate the state of the resources in the resource collection. This isn’t common in most recipes. It’s required for certain kinds of development patterns like “wrapper” cookbooks. As an example, I’m going to modify the resource object so I don’t have to log into the system again and run apt-get remove vim, to show the next section. First, I’m going to create a local variable in the context of the recipe. This is just like any other variable in Ruby. For its value, I’m going to use the #resources() method to look up a resource in the resource collection. The return value is the package resource object: (#class is a method on the Ruby Object class that returns the class of the object) To remove the vim package, I use the #run_action method (available to all Chef::Resource subclasses), specifying the :remove action as a symbol: There is no additional debug to display. Chef will run apt-get remove vim to converge the resource with this action. Load Current Resource Redux Now that the package has been removed from the system, what happens if we run Chef again? Well, Chef is convergent, and it takes idempotent actions on the system to ensure that the managed resources are in the desired state. That means it will install the vim package. We’ll see some familiar messages here about the version, then: This is load_current_resource working as expected. As we can see from the apt-cache policy output, the package is not installed, and as the action to take is :install, Chef will do what we think: This should be familiar to anyone that uses Debian/Ubuntu, it’s standard apt-get install output. Of course, this is a development system so I have some cruft, but we’ll ignore that ;). If we run_chef again, we get the output we saw in the original example in this post: Planet Clojure Ninety-Nine Haskell Problems [Euler/Clojure too] Ninety-Nine Haskell Problems From the webpage: These are Haskell translations of Ninety-Nine Lisp Problems, which are themselves translations of Ninety-Nine Prolog Problems. Also listed are: Naming isn’t the only hard problem in computer science. The webpage points out that due to gaps and use of letters, there are 88 problems and not 99. If you want something a bit more challenging, consider the Project Euler problems. No peeking but there is a wiki with some Clojure answers, http://clojure-euler.wikispaces.com/. Enjoy! I first saw this in a tweet by Computer Science. /r/netsec Perverting Embedded Devices - ZKSoftware Fingerprint Reader (Part I) ~ Infobyte Security Research Labs StackOverflow Grails 1.3.9 and grails-melody: javamelody.jar not packaged when building war with OpenJDK 1.6 We are maintaining a webapp developed in Grails 1.3.9. For monitoring performance, the app had grails-melody 1.21 plugin installed. It seems that the plugin is not available in the repositories any more for grails 1.3.x. I downloaded it from google-code as suggested in the documentation. Another post in stackoverflow suggests that zipped plugins can be put in lib and then referenced from BuildConfig.groovy. plugins { runtime ":hibernate:1.3.9" build ":tomcat:1.3.9" compile ":dojo:1.6.1.17" // Downloaded from // https://code.google.com/p/javamelody/downloads/list?can=1&q=grails // Installed from lib // http://stackoverflow.com/questions/15751285/whats-the-correct-way-to-install-a-grails-plugin-from-a-zip-file compile ":grails-melody:1.21" }  I did that and this procedure worked fine when building war file with Oracle JDK 7 (on Ubuntu 14.04). I had to rename grails-grails-melody-1.21.zip to grails-melody-1.21.zip so that it was found.  java -version java version "1.7.0_65" Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)  During the build, melody's dependencies were loaded at the beginning of the build process, when grails file were copies as well: ... Downloading: /home/matejk/devel/grails/grails-1.3.9/lib/servlet-api-2.5.jar ... Download complete. Downloading: /home/matejk/devel/grails/grails-1.3.9/lib/jsp-api-2.1.jar ... Download complete. Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.pom ... Download complete. Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.pom.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.pom ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.pom.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.pom ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.pom.sha1 ... Download complete. Downloading: /home/matejk/devel/grails/grails-1.3.9/lib/groovy-all-1.7.8.jar ... Download complete. Downloading: /home/matejk/devel/grails/grails-1.3.9/lib/commons-beanutils-1.8.0.jar ... Download complete. ... Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.jar ... Download complete. Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.jar.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.jar ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.jar.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.jar ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.jar.sha1 ... Download complete. Downloading: /home/matejk/devel/grails/grails-1.3.9/lib/aspectjweaver-1.6.8.jar ... ...  Resulting war file had javamelody, jrobin and itext jars in WEB-INF/lib. However, the requirement is to build the app with JDK 1.6 on another machine (Jenkins) where clean checkout of sources is done for every build. java -version java version "1.6.0_31" OpenJDK Runtime Environment (IcedTea6 1.13.3) (6b31-1.13.3-1ubuntu1) OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)  Download of jars happened later in the build process: Executing hibernate-1.3.9 plugin post-install script ... Plugin hibernate-1.3.9 installed Installing zip /var/lib/jenkins/.ivy2/cache/org.grails.plugins/grails-melody/zips/grails-melody-1.21.0.zip... ... [mkdir] Created dir: /var/lib/jenkins/workspace/etermin-2.4/target/projects/etermin-2.4/plugins/grails-melody-1.21 [unzip] Expanding: /var/lib/jenkins/.ivy2/cache/org.grails.plugins/grails-melody/zips/grails-melody-1.21.0.zip into /var/lib/jenkins/workspace/etermin-2.4/target/projects/etermin-2.4/plugins/grails-melody-1.21 Installed plugin grails-melody-1.21 to location /var/lib/jenkins/workspace/etermin-2.4/target/projects/etermin-2.4/plugins/grails-melody-1.21. ... Resolving plugin JAR dependencies ... Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.pom ... Download complete. Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.pom.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.pom ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.pom.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.pom ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.pom.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.jar ... Download complete. Downloading: http://repo1.maven.org/maven2/net/bull/javamelody/javamelody-core/1.44.0/javamelody-core-1.44.0.jar.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.jar ... Download complete. Downloading: http://repo1.maven.org/maven2/com/lowagie/itext/2.1.7/itext-2.1.7.jar.sha1 ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.jar ... Download complete. Downloading: http://repo1.maven.org/maven2/org/jrobin/jrobin/1.5.9/jrobin-1.5.9.jar.sha1 ... Download complete.  However, when creating war with that JDK, javamelody, jrobin and itext jars are not packaged. Consequently deployment and startup of webapp fails. SEVERE: Error configuring application listener of class net.bull.javamelody.SessionListener java.lang.ClassNotFoundException: net.bull.javamelody.SessionListener at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4153)  • Is the procedure to install plugin from local directory correct? • Any ideas how to resolve this problem to make jars to be packaged into war? Thanks, Matej EDIT: After removing grails files in ~/.grails/1.3.9/ the behaviour with JDK 6 and JDK 7 is the same: jar files are not packaged and deployment fails. CompsciOverflow Min-max selection sort Is there already modified version of selection sort that works like this pseudocode: a // the array, index starts at 1 (not zero-based) n = len(a) for x in 1 to n big = x small = x last = n-x+1 for y in x+1 to last if a[big] < a[y] then big = y if a[small] > a[y] then small = y if big <> small if small <> x then swap(a[small],a[x]) if big <> last then swap(a[big],a[last])  if there is such an algorithm, what is the name of it? /r/compsci Ghost out of Shell Matt Might Understand and implement laziness with examples in Scala, JavaScript, Swift and Racket At first glance, laziness seems to be about efficiency. In computing, laziness means delaying the computation of a value until the value is necessary. (And, if the value is never necessary, it is never computed.) In reality, laziness is more about expressiveness: laziness allows the expression of programs that would otherwise not terminate. In some languages (like Scala, Swift and Perl), laziness also makes it possible to add new constructs to a language without using macros. Since most languages do not provide support for laziness, this article explores how to implement laziness in languages that support closures. We’ll look at manual transformations from languages that have native support for laziness (Scala) into (1) languages that do not support laziness (JavaScript) and (2) languages that have partial support (Swift). We’ll also use macro-generating macros in Racket to implement native and transparent support for multiple kinds of laziness in a strict language. Click here to read the rest of the article Lobsters Blazing Fast HTML: Virtual DOM in Elm /r/scala Vote for semantic diff support for Scala (if that's what you want) StackOverflow Build scatter-gather in akka I want to build actor chain with scatter-gather pattern My flow looks like this: a1=>a2=>a3=>a4=>a5  When I creating actors I use following code: val a5: ActorRef = actorSystem.actorOf(Props(new A5)) val a4: ActorRef = actorSystem.actorOf(Props(new A4(a5, otherArgs4)) val a3: ActorRef = actorSystem.actorOf(Props(new A3(a4, otherArgs3))) val a2: ActorRef = actorSystem.actorOf(Props(new A2(a3))) val a1: ActorRef = actorSystem.actorOf(Props(new A1(a2)))  Can I create actors chain in smarter and more configurable way? Access @ApiOperation attributes in Swagger ApiAuthorizationFilter I have a Jax-RS webservice, which I document using Swagger. Now I want to check the attributes from the @ApiOperation annotation of each method to do some filtering. @GET @ApiOperation(value = "a description", response = String.class, hidden=true) public static Response get(){}  I created an ApiAuthorizationFilterImpl according to this example: https://github.com/wordnik/swagger-core/blob/master/samples/java-jaxrs/src/main/java/com/wordnik/swagger/sample/util/ApiAuthorizationFilterImpl.java#L60 I overwrote the isOperationAllowed method: public boolean isOperationAllowed(Operation operation, ApiDescription api, Map<String, List<String>> params, Map<String, String> cookies, Map<String, List<String>> headers) { System.out.println("params: "+params.keySet()); //<--params is empty }  Now, when I am trying to access params, it is empty. On the opposite, I can print the parameters from operation.parameters() but I also can't access them, as I get a Scala list with parameters. The filter in general is access when starting the Swagger-UI, so it seems to be correctly passed in my web.xml. Is this a silly mistake from my side, or how will I be able to access the parameters? As a workaround I also wouldn't mind to access parameters from @ApiModelProperty annotation. In sbt can't add or override maven URL resolvers with params I'm trying to override the default external-resolvers with maven resolvers of type http:/.....?key=value. I tried to add to build.sbt the following with no luck of solving my problem: val repo1="Repo1" at "http://.....?key=value" val repo2=Resolver.url("Repo2", url("http://....."))( Patterns(Seq("[organisation]/[module](_[scalaVersion])(_[sbtVersion])/[revision]/[module](_[scalaVersion])(_[sbtVersion])-[revision](-[classifier]).pom?key=value"), Seq("[organisation]/[module](_[scalaVersion])(_[sbtVersion])/[revision]/[artifact]-[revision](-[classifier]).[ext]?key=value"), true) ) externalResolvers:=Seq(repo1, repo2)  repo1, of course, doesn't work because the the params are embedded inside the URL and not at the end after the question mark (e.g., http:/.....?key=value/org/mockito/mockito-all/1.9.5/mockito-all-1.9.5.pom). While repo2 seems to have a problem analyzing the the returned pom file: problem while downloading module descriptor: https:/...../org/mockito/mockito-all/1.9.5/mockito-all-1.9.5.pom?key=value: invalid md5: expected=<?xml computed=8cde60b7d723d15aeb5e59aaf033351f (821ms) It seems to me that sbt is considering repo2 as a repo of type ivy and not as a maven one. I tried the above with sbt 0.13.0 and 0.13.5. Any idea how to solve this issue and support url with params? Create a service which returns a Future [on hold] I have been dealing with Play! Framework recently. I noticed some API support asynchronous services. For example MongoDB has ReactiveMongo which supports asynchronous API. I am interested in creating my own asynchronous services, however could not find any reference online. How would you implement asynchronous services? Or is there a good materials I can study on? Edit: My question is, what does it take a service to be non-blocking or asynchronous? I want to create a service which will not block the caller, but allow caller to wait on the future. /r/netsec What is memory safety? Halfbakery Universal 'Like' Button (0.5) Portland Pattern Repository How Goedel Sentences Work (by hide.worldspan.com 29 hours ago) Planet Scala Scala for Java Developers A few weeks ago I was asked to review a book called "Scala for Java Developers". Being preparing some Scala lessons to share at work it made lots of sense to agree. Finding a good way to introduce Java and .NET developers to Scala is hard. If someone is interested in learning a new language it's easy, just give them the Programming in Scala book. The challenge I'm up against is making a case for actually spending time learning it at all. What caught my eye a few years ago was all the expressiveness and succinctness I could achieve by making the leap. My expectations were to find a book that explained how to translate all the Java knowledge into Scala while learning the interesting new features one could discover making such a transition. The books was not exactly what I expected but that is not a bad thing. The first chapter introduces the reader to the Activator and drops them into the REPL. It shows good examples of how easy it is to declare classes, play around with collections and express basic stuff. That is what I expected to find but then it picks up speed. It continues on the next chapter by creating a Java web service and exemplifying how to migrate it to Scala piece by piece. Then it goes on to IDEs, SBT, mentions many useful SBT plugins, worksheets and even a writing a REST client. Followed by a tour of all the testing frameworks available on Scala land. Moving to Play Framework and then some debugging. Halfway through the book most of what regular developer faces on a day to day basis has been touched upon. Scala syntax is explained almost by accident, just as a necessity to show what can be done. It doesn't teach the language but rather provides examples of interesting and most likely new and simpler ways to do stuff a Java developer is used to. The rest of the book continues on at full speed with databases, webservices, xml, json, CRUDs and concurrency. I was surprised it even tackles Iteratees to illustrate reactive web applications like chat rooms. It even mentions Scala.js! Overall it shows many libraries and tools used in the Scala ecosystem. It takes on many of the common problems most people are trying to solve at work. I think it can appeal to Spring, JavaEE or Rails users and similar. It provides a bird's eye view of the frameworks available while forcing the reader to actually try stuff out immediately in the REPL. Readers will get a feeling for what Scala enables and will surely go out looking for more. My conclusion is if you are already into Scala and learning on your own or with other books then it is not for you. I recommend the book as a teaser for programmers who don't know Scala at all or barely and want to discover what it is about skipping to the fun part. Also, for someone like me who is trying to get other people interested it provides many ideas and examples that are simple to setup and play with. Having more Scala resources is good and this book fills a spot that needed to be filled. /r/freebsd Automatic decryption of GELI disks with encrypted keys and passphrases stored on a remote system. I always liked the idea of more of a deadmans switch style of encryption security, so I created an rc script that automatically retrieves geli keys and passphrases from a remote system via ssh. You can put the key and the passphrase on different systems for even more security, and the key file and passphrase file can be encrypted themselves so they can be stored on an untrusted server. You could use this to encrypt a disk on a VPS for example and have it retrieve the keys from your local server. So long as your local server is up your VPS can reboot unattended. Or the other way around, you could store the encryption keys on a VPS or even a shell host as long as you have scp access. Even busybox serves scp, so you could store a key or passphrase file on an embedded router or an android phone if you want. Hopefully others find this useful, I think it strikes a nice balance of security and convenience. https://github.com/clinta/geliUnlocker submitted by FakingItEveryDay [link] [4 comments] Lobsters Towards understanding Haskell’s monomorphism restriction QuantOverflow How is PnL calculated In Fixed Income, I know that bonds PnL are evaluated depending on where the price lies on price/yield curve at the end of the day, compared to where it started from at beginning of the day. The portfolio of bonds will have a specific DV01, which will be used to compute the PnL. Can someone tell me if this is right or is there something more? For equities it should be just a simple sum of stock prices at the end of day vs beginning of day? Is this right? Lobsters When Programming Goes Bad - Debugging the Impossible StackOverflow Default implementation for generic method Suppose I have a trait Foo Also I have a trait that uses trait Foo in type parameter trait Bar[T <: Foo] { def bar: T = new Foo {} //doesn't compile }  I want to provide default implementaition in trait Bar for bar method. I need this because there are several implementations that do not need to override bar method(they are just happy with Foo), but also there are some implemenations that do need to provide specific implemetaions of Foo in bar method. Is this possible? Lobsters Building Cordova Apps with the Verified Plugins Marketplace Overcoming Bias Tegmark’s Vast Math I recently had a surprise chance to meet Max Tegmark, and so I first quickly read his enjoyable new book The Mathematical Universe. It covers many foundations of physics topics that he correctly says are unfairly neglected. Since I’ve collected many opinions on foundation of physics over decades, I can’t resist mentioning the many ways I agree and disagree with him. Let me start with what Tegmark presents as his main point, which is that the total universe is BIG, almost as big as it could possibly be. There’s a vast universe out there that we can’t see, and will never see. That is, not only does space extent far beyond our cosmological horizon, but out there are places where physics sits in very different equilibria of fundamental physics (e.g., has a different number of useful dimensions), and nearby are the different “many worlds” of quantum mechanics. Furthermore, and this is Tegmark’s most unique point, there are whole different places “out there” completely causally (and spatially) disconnected from our universe, which follow completely different fundamental physics. In fact, all such mathematically describable places really exist, in the sense that any self-aware creatures there actually feel. Tegmark seems to stop short, however, of David Lewis, who said that all self-consistent possible worlds really exist. Tegmark’s strongest argument for his distinctive claim, I think, is that we might find that the basic math of our physics is rare in allowing for intelligent life. In that case, the fact of our existence should make us suspect that many places with physics based on other maths are out there somewhere: Why do the five arrows in the bottom panel conspire to allow any habitable range of Higgs properties? This could well be a fluke: five random arrows would allow some range with 19% probability, so we need only invoke a small amount of luck. … However, it’s perfectly plausible that further physics research could uncover more striking fine-tuning of this discrete type with, say, ten or more arrows conspiring to allow a habitable range for some physical parameter or parameters. And if this happens, then we can argue … that this is evidence for the existence … of other universes where the laws of physics are different, giving quite different requirements for life! I accept that this consideration would modestly push us in this direction, just as the existence of physics parameters fine-tuned for life modestly pushes us to believe that other spatial regions out there have different values of those parameters. I accept these because I accept the self-indication principle in indexical inference, which says that the info that you exist should increase your belief in possible worlds where you might have existed, in proportion to the number of slots where you could have existed. However, it isn’t clear that Tegmark accepts the self-indication principle, since he also says that you should believe in many worlds quantum mechanics if you survive many rounds of quantum suicide, which is where you randomly suicide depending on a quantum event. The self-indication principle does not say that seeing this should increase your belief in many worlds, as there aren’t substantially more slots for you to occupy in scenarios where quantum suicide attempts fail. Since both many worlds and stochastic quantum mechanics predict exactly the same conditional probabilities of observations given quantum suicide attempts, such data can’t distinguish them. So I wouldn’t see surviving quantum suicide as further evidence for many worlds. Tegmark also argues that we should reject a description of reality that has math plus extra “baggage.” Apparently, if there are many possible math descriptions of universes, then it would be extra “baggage” for the universe to point to only one math and say “that’s the math that really exists.” This is pretty close to the argument of David Lewis. But Tegmark also seems to accept that there is some “measure” over all these maths, and that the reason to take some actions over others is to favorably change that measure. Yet this measure also seems to also be an extra “baggage” over and beyond the math descriptions themselves. The main difference I see between adding a “this exists” pointer and adding a measure over the space of maths is that the later approach has more slots for me to exist, and so is favored by self-indication. Tegmark does say that we struggle to find reasonable measures for many physics problems, and as a result we don’t actually know what inflation implies. He suspects this is due to our over-eager embrace of the concept of infinity. But Tegmark also follows Everett in using infinity to derive the Born probability rule in quantum mechanics. Everett noted that the relative measure of Born-rule-deviant worlds approached zero after an infinity of measurements, and thus the final state is “as if” those worlds didn’t exist. To get the same effect, Tegmark instead considers a superposition of an infinity of spatial regions out there containing the same exact observer. But this trick wouldn’t work with a finite set of such regions, and I have my doubts that it is fair to treat such regions is if they hadn’t decohered. (My approach to deriving the Born rule in many worlds stays with strictly finite everything.) Tegmark also seems to think that a similar approach explains the second law of thermodynamics, but that doesn’t at all make sense to me. The main thing to explain there is why our best measure for predicting the past is pretty much the inverse of our best measure for predicting the future. I didn’t see Tegmark addressing that at all. (Btw, Sean Carroll recently mostly rejected the solution he wrote a book elaborating.) My last disagreement with Tegmark is that he sees the simulation argument as self-destructing because since we know nothing about the universe that might simulate us, it could also be simulated by another universe, which is a “a reductio ad absurdum.” But I think it quite reasonable to put substantial weight there being real costs of computation in the base universe, which means they would tend to limit the ability of universes they simulate to spawn yet further universes. Which makes it less likely that we are a second level simulation, relative to a first level one. Let me conclude with some of the many ways I agree with the book: • We reject or accept total theories, not places. If parallel universes are implied by our best theories, we should believe in them. • It is vitally important to figure out good physics measures, and good physics explanations for them. • I also prefer to reason about theories without infinities, when nearly as plausible on other grounds. Take a limit to infinity as the very last theory step, if possible. • Our brains aren’t doing much quantum computing. • Quantum mechanics seems random because we observers are splitting; the universe is deterministic. • Foundations of physics gets unfairly neglected. People who do it have to hide it to save their careers. • The world is hurting itself by not coordinating to reduce existential risk. We have a vast great potential future if we don’t kill ourselves. • We don’t need a new understanding of consciousness to solve foundations of physics problems. “Consciousness is the way information feels when being processed in certain complex ways.” Saying that our math “exists” is pretty much the same as saying that the stuff that fits our math is the stuff that “feels” when arranged in certain ways. StackOverflow How to run Scala files on a web server This could be either an incredibly easy or horrifically complicated question; but I've started writing code in Scala, and want to run said code on a web server. There's a few questions that I need answering really, and I apologise for my complete lack of knowledge on the subject, web servers scare me (I'm a PHP developer so all I need to do there is upload to any linux apache server): • How to save and upload files (I know how to use ftp software, do I need to do anything from Eclipse?) • What to do to my server to run the files (at the moment I just have a linux apache server from fasthosts, is that enough?) Any other advice would be so greatly appreciated; and the less jargon the better - the whole thing blags me a little. UPDATE: To put the sort of work I'm doing into context, I'm making a rather large, social-media style site that'll have a very large amount of small, regular posts/updates. Also thanks for the answers so far! Seems like i need to do all of the following: • Learn Scala (I can write in PHP and Java so should be okay) • Install Apache tomcat on my server somehow • Install Lift (and then either Maven or the SBT) • Upload the whole thing? What do I do about databases?! I need something really serious, with MySQL be okay? Urgh this goes on and on... /r/netsec Stealing unencrypted SSH-agent keys from memory QuantOverflow Risk Parity portfolio construction If I would like to construct a fully invested long only portfolio with two asset classes (Bonds B and Stocks S) based on the concept of 'risk parity' the weights W of my portfolio would be the following: Then the weight of the bonds W_B = \textrm{Vol}(S)/[\textrm{Vol(S)}+\textrm{Vol(B)}] and the weights of the stocks W_S = 1 - W_B. Based on this I am going to overweigh the low volatility asset and underweight the high volatility asset. My question is: How to calculate the weights for a portfolio with multiple asset classes, 5 for example, so that each asset class will have the same volatility and contribute the same amount of risk into my portfolio. From historical data I can extract the volatility of each asset class and the correlation between them. Fefe Die Russen sagen, ein ukrainischer Su-25 sei zur Abschusszeit ... Die Russen sagen, ein ukrainischer Su-25 sei zur Abschusszeit nahe an MH17 rangeflogen. Leider ist Su-25 kein Abfangjäger sondern ein Erdkampfflugzeug, das soll Bodentruppen unterstützen. Der kommt bis 7km hoch, die MH17 flog 10. Aber abgesehen davon, voll die solide Theorie!1!! Update: Die Pressekonferenz soll wohl ansonsten ganz lustig gewesen sein; die Russen haben gemeint, hey, die Amis haben ja umfangreiche Telemetriedaten von dieser Liste an Satelliten hier, die das hätten sehen müssen. Die sollen doch einfach mal die Daten auf den Tisch legen!1!! Update: Ah, da steht jetzt mehr. Ja, man kann in eine Su-25 auch Luft-Luft-Raketen tun. RT schreibt, Su-25 käme bis 10 km hoch. Abgesehen davon hat die ukrainische Luftwaffe auch Abfangjäger wie den Su-27, die können dann bis 19km rauf. Oh, und die Russen legen gleich mal Aufklärungsfotos der Buk-Stellungen der Ukrainer bei und meinen, die MH17 sei in deren Range gewesen. StackOverflow Implementing Receiver in SPARK I have been trying to implement receiver for SPARK 0.9 . I have captured packets using Jnetpcap library and need to pass it to spark in Scala. Is it enough to write the capturing part of the packet in "def receive()" method? Edit: Here is code from this link that captures packets using the Jnetpcap library: import java.util.Date import java.lang.StringBuilder import org.jnetpcap.Pcap import org.jnetpcap.packet.PcapPacket import org.jnetpcap.packet.PcapPacketHandler object PacketCapture1 { def main(args: Array[String]){ val snaplen = 64 * 1024 // Capture all packets, no trucation val flags = Pcap.MODE_PROMISCUOUS // capture all packets val timeout = 10 * 1000 //val errbuf = new StringBuilder() val jsb = new java.lang.StringBuilder() val errbuf = new StringBuilder(jsb); val pcap = Pcap.openLive("eth0", snaplen, flags, timeout, errbuf) if (pcap == null) { println("Error : " + errbuf.toString()) } println(pcap) val jpacketHandler = new PcapPacketHandler[String]() { def nextPacket(packet: PcapPacket, user: String) { println("Received packet at %s caplen=%4d len=%4d %s\n", new Date(packet.getCaptureHeader.timestampInMillis()), packet.getCaptureHeader.caplen(), packet.getCaptureHeader.wirelen(), user) } } pcap.loop(30, jpacketHandler, "jNetPcap works!") pcap.close() } }  How to implement spark receiver for the packets captured using this code? Lobsters What is memory safety? CompsciOverflow Turing Machine that Determines whether an equal amount of 1's and 0's have been scanned [on hold] Design a program for a Turing machine. Create a decision procedure for the set of binary strings which contain the same number of 0's and 1's. Please help! Thanks. I am a beginner student and have no idea where to start. So sorry for how elementary this is. Lambda the Ultimate Forum Depth-first search in APL The latest video in Dyalog's library - Depth-First Tree-Search in APL - is now available (https://www.youtube.com/watch?v=DsZdfnlh_d0) The classic depth-first search algorithm is explored using APL, a simple and concise array notation with high-order functions. The presentation highlights APL's incremental development style, using simple steps and culminating in a purely functional solution for the N-Queens problem. The informative style of this presentation brings clarity to this advanced topic, making it accessible even to those who are still near the start of their APL journey. Once you've seen the video, why not examine the code used in greater detail and try the expressions for yourself in the online tutorial at http://tryapl.org/ (Learn tab > Depth-first search) StackOverflow "failure in processEncodedAnnotation" using Jackson with ScalaModule on Android I use Scala 2.11.0 with sbt and android-sdk-plugin together with Jackson as the Json Marshaller. When the app tries to deserialize from Json to my Scala case class the following exception occurs on the device: W/dalvikvm﹕ WARNING: could not find annotation member bytes in Lscala/reflect/ScalaSignature; E/dalvikvm﹕ Failed creating annotation element (m=0x41a575c0 n=0x41a575e0 a=0x0 r=0x0) W/System.err﹕ java.lang.RuntimeException: failure in processEncodedAnnotation W/System.err﹕ java.lang.RuntimeException: failure in processEncodedAnnotation W/System.err﹕ at java.lang.Class.getDeclaredAnnotations(Native Method) W/System.err﹕ at com.fasterxml.jackson.databind.introspect.AnnotatedClass.resolveClassAnnotations(AnnotatedClass.java:308) W/System.err﹕ at com.fasterxml.jackson.databind.introspect.AnnotatedClass.getAnnotation(AnnotatedClass.java:173) W/System.err﹕ at com.fasterxml.jackson.databind.introspect.JacksonAnnotationIntrospector.findAutoDetectVisibility(JacksonAnnotationIntrospector.java:140) W/System.err﹕ at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.<init>(POJOPropertiesCollector.java:111) W/System.err﹕ at com.fasterxml.jackson.module.scala.introspect.ScalaPropertiesCollector.<init>(ScalaPropertiesCollector.scala:15) W/System.err﹕ at com.fasterxml.jackson.module.scala.introspect.ScalaClassIntrospector.constructPropertyCollector(ScalaClassIntrospector.scala:24) W/System.err﹕ at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:142) W/System.err﹕ at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forDeserialization(BasicClassIntrospector.java:81) W/System.err﹕ at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forDeserialization(BasicClassIntrospector.java:11) W/System.err﹕ at com.fasterxml.jackson.databind.DeserializationConfig.introspect(DeserializationConfig.java:547) W/System.err﹕ at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:325) W/System.err﹕ at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:263) W/System.err﹕ at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:243) W/System.err﹕ at com.fasterxml.jackson.databind.deser.DeserializerCache.hasValueDeserializerFor(DeserializerCache.java:193) W/System.err﹕ at com.fasterxml.jackson.databind.DeserializationContext.hasValueDeserializerFor(DeserializationContext.java:345) W/System.err﹕ at com.fasterxml.jackson.databind.ObjectMapper.canDeserialize(ObjectMapper.java:2072) W/System.err﹕ at org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.canRead(MappingJackson2HttpMessageConverter.java:103) W/System.err﹕ at org.springframework.web.client.RestTemplateAcceptHeaderRequestCallback.doWithRequest(RestTemplate.java:542) W/System.err﹕ at org.springframework.web.client.RestTemplateHttpEntityRequestCallback.doWithRequest(RestTemplate.java:593) W/System.err﹕ at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:474) W/System.err﹕ at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:439) W/System.err﹕ at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:415) W/System.err﹕ at net.gutefrage.android.rest.ScalaRestTemplate$$anonfun$exchange$1.apply(ScalaRestTemplate.scala:22) W/System.err﹕ at net.gutefrage.android.rest.ScalaRestTemplate$$anonfunexchange1.apply(ScalaRestTemplate.scala:22) W/System.err﹕ at scala.util.Try.apply(Try.scala:161) W/System.err﹕ at net.gutefrage.android.rest.ScalaRestTemplate$$anonfun$genericRestCallForEntity$1.apply(ScalaRestTemplate.scala:31) W/System.err﹕ at net.gutefrage.android.rest.ScalaRestTemplate$$anonfungenericRestCallForEntity1.apply(ScalaRestTemplate.scala:31) W/System.err﹕ at scala.concurrent.impl.FuturePromiseCompletingRunnable.liftedTree11(Future.scala:24) W/System.err﹕ at scala.concurrent.impl.FuturePromiseCompletingRunnable.run(Future.scala:24) W/System.err﹕ at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112) W/System.err﹕ at java.util.concurrent.ThreadPoolExecutorWorker.run(ThreadPoolExecutor.java:587) W/System.err﹕ at java.lang.Thread.run(Thread.java:841)  [Exception: DB plugin is not registered.] package models import play.api._ import play.api.mvc._ import anorm._ import anorm.SqlParser._ import play.api.db.DB import play.api.Play.current import scala.language.postfixOps case class User(name:String) object User { def insert(name:String) : Boolean={ DB.withConnection { implicit connection => val result:Boolean= SQL("insert into user(name,username,password) VALUES('"+name+"','"+name+"','"+name+"')").execute() return result } } }  "In above program an exception is occured named "[Exception: DB plugin is not registered.]". how to fix it" Twilio Proxy: Howto create HTTPS Rest Spray Server I found the shadow proxy twilio: https://github.com/twilio/shadow With this proxy I can forward an incoming request on the one hand to a true server and on the ohter hand to a shadow server. The Twilio proxy is using Spray for REST to accomplish this task. If everything is sent via HTTP, I could test it successfully. But I like to send all the data via TLS. My question is: How to create a HTTPS-Server with Spray? The full code you can find here: https://github.com/twilio/shadow/blob/scala/src/main/scala/com/twilio/shadow/ProxyService.scala Here's the code excerpt from ProxyService.scala how the Twilio proxy creates the HTTP Proxy: def buildNewRequest(request: HttpRequest, config: ShadowServerConfig): HttpRequest = { val newUri = Try({ val origUri = URI.create(request.uri) val queryParams = overrideValues(request.queryParams, config.queryOverride) new URI(origUri.getScheme, origUri.getUserInfo, origUri.getHost, origUri.getPort, origUri.getPath, marshal(FormData(queryParams)).right.get.asString, origUri.getFragment).toASCIIString }).getOrElse(request.uri) val newEntity = request.header[HttpHeaders.Content-Type] match { case Some(_) => { Try({ val form = request.entity.as[FormData].right.get val formParams = form.copy(fields = overrideValues(form.fields, config.formOverride)) marshal(formParams).right.get }).getOrElse(request.entity) } case None => request.entity } HttpRequest(request.method, newUri, request.headers, newEntity, request.protocol) }  The TLS support is described here: http://spray.io/documentation/1.1-M7/spray-io/predefined-stages/#ssltlssupport I do not know how to send a HTTPS-Request object. Bringing it in an example, it should look like this: The client sends an request to the url of the shadow proxy: "https://localhost:8000" and the shadowProxy forwards it to a) the true-server "https://localhost:1234" b) and to the shadow-server "https://localhost:2345"  Thanks for any help. Ansible Django Module Alternative Python Interpreter I'm writing an ansible playbook for deplying a django app and as part of the process I'd like to run the staticcollect command. The issue I'm having is that the remote server has two python interpreters, one Python2.6 and one Python2.7, with Python2.6 being the default. When I run the playbook it runs using the Python2.6 interpretor and I need it to run against the Python2.7 interpretor. Any idea on how this can be acheived? My playbook is as follows: - hosts: xxxxxxxxx vars: hg_branch: dmv2 django_dir: /opt/app/xxxx conf_file: /opt/app/conf/xx_uwsgi.ini django_env: STATIC_ROOT: /opt/app/serve/static remote_user: xxxxxx tasks: - name: Update the hg repo command: chdir=/opt/app/xxxxx hg pull -u --rev {{hg_branch}} - name: Collect static resources environment: django_env django_manage: command=collectstatic app_path={{django_dir}} - name: Restart the django service command: touch {{conf_file}} - name: check nginx is running service: name=nginx state=started  AWS AWS Week in Review - July 14, 2014 Let's take a quick look at what happened in AWS-land last week:  Monday, July 14 We launched Annual Pricing for AWS Marketplace Products. We announced Time to Live (TTL) Control for Amazon SNS. We announced that you can now Store and Monitor OS and Application Log Files with Amazon CloudWatch. We published Episode #93 of the AWS Podcast . I interviewed David Etue of SafeNet. Tuesday July 15 We announced General Availability and a Service Level Agreement for Amazon RDS for PostgreSQL. The AWS Application Management Blog reviewed Customer Highlights from Around the Web. A guest post on the AWS Startup Collection talked about Distributed Job Scheduling for AWS. Wednesday, July 16 As part of an AWS Support API Update, we added support for creation of attachments and lightweight monitoring of support cases. We announced that Amazon Kinesis is now Available in Asia Pacific (Asia Pacific Tokyo, Asia Pacific Singapore, and Asia Pacific Sydney). We announced that Simple Queue Service and Auto Scaling API calls are now available in CloudTrail. We announced Enhanced Password Management and Credential Reports for IAM. This topic was covered in depth in the AWS Security Blog in the post New IAM Features: Enhanced Password Management and Credential Reports. We launched the AWS Big Data Blog . The first post showed you how to Build a Recommender with Apache Mahout on Amazon Elastic MapReduce (EMR). Thursday, July 17 The AWS Big Data Blog talked about Powering Gaming Applications with Amazon DynamoDB. The Amazon Simple Email Service Blog talked about Bounces to Domains That You Have Verified. The AWS PHP Developer Blog announced the Launch of Version 2.6.12 of the AWS SDK for PHP. The AWS Startup Collection published Part 2a of an article on Internet of Things. A cool story on the Twilio (powered by AWS) Blog talked about how A Load Test Saved A Florida County Election Call Center In 4-lines of Code. Friday, July 18 The AWS Windows and .NET Developer Blog talked about Subscribing Websites to Amazon SNS Topics. The AWS Java Blog talked about Pausing and Resuming Transfers Using Transfer Manager. The AWS Ruby Development Blog talked about Response Paging in Version 2 of the AWS SDK for Ruby. We introduced new AWS Customer Success Stories from Condé Nast, FINRA, Novartis, and Siemens. We updated the EC2 Public IP Address Ranges, adding 917,496 addresses, bringing the total to 8,700,728. AWS Marketplace added new products including Fortinet FortiWeb-VM, NGINX Streaming Media Server, and Nessus Enterprise from Tenable. Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed. -- Jeff; StackOverflow Translating "Why Functional Programming Matters" into Haskell For cultural and intellectual enrichment, I've decided to learn a bit of Haskell. I've been reading Hughes' "Why Functional Programming Matters" and am trying to translate its code into true Haskell. I've attached some of my attempt below (for the numerical parts of the paper; the alpha-beta algorithm is even more interesting but I'd also have to write a game evaluator from scratch!). At this point it's been more of an exercise in Haskell syntax than anything else. I've already done simple things like translate repeat into the native Haskell iterate, translate a few functions that used a lot of parentheses to function composition (making them more point-free in the process), etc. But my code certainly works, and I wonder if it's sufficiently "Haskell-ish". Can any experts out there give me some hints? -- 4.1 Newton-Raphson square roots next n x = (x + n/x)/2.0 -- -- this is "iterate::(a->a)->a->[a]" -- repeat f a = a : iterate f (f a) within eps (a:b:rest) = if abs(a-b) <= eps then b else within eps (b:rest) sqroot a0 eps n = within eps (iterate (next n) a0) relative eps (a:b:rest) = if abs(a-b) <= eps*abs(b) then b else relative eps (b:rest) relativesqrt a0 eps n = relative eps (iterate (next n) a0) -- 4.2 numerical differentiation easydiff f x h = (f (x+h) - f x) / h differentiate h0 f x = map (easydiff f x) (iterate (/2) h0) -- diff1a h0 eps f x = within eps (differentiate h0 f x) diff1 h0 eps f = within eps . differentiate h0 f elimerror n (a:b:rest) = (b*(2**n)-a)/(2**n-1) : elimerror n (b:rest) -- need fromIntegral to make a non-integer out of the Int which comes out of round order (a:b:c:rest) = fromIntegral (round (logBase 2 ((a-c)/(b-c)-1))) improve s = elimerror (order s) s --diff2a h0 eps f x = within eps (improve (differentiate h0 f x)) diff2 h0 eps f = within eps . improve . differentiate h0 f --super s = map second (iterate improve s) -- how can we make this point-free? super :: (RealFrac t, Floating t) => [t] -> [t] -- w/o this it wants to be [double]->[double] super = map second . iterate improve -- second (a:b:rest) = b second = head . tail diff3 h0 eps f = within eps . super . differentiate h0 f -- 4.3 integration easyintegrate f a b = (f a + f b)*(b-a)/2 -- addpair becomes (uncurry (+)) integrate f a b = integ f a b (f a) (f b) integ f a b fa fb = (fa+fb)*(b-a)/2 : map (uncurry (+)) (zip (integ f a m fa fm) (integ f m b fm fb)) where m = (a+b)/2 fm = f m -- test: following should be about pi approxpi eps = within eps (improve (integrate (\x -> 4/(1+x*x)) 0 1)) superpi eps = within eps (super (integrate (\x -> 4/(1+x*x)) 0 1)) -- is there any way to keep track of the number of iterations? state monad, but seems like a lot of work...\  Scala: combine elements in same position of two lists I've two lists and I want to obtain another list combining, in a tuple, elements in same position. An example: val x = List(1,2,3) val y = List(4,5,6) result = List((1,4),(2,5),(3,6))  Check for acceptance of type, rather than value, with isDefinedAt I have a case where I want use isDefinedAt to check if a partial function accepts a type, rather than a specific value. val test: PartialFunction[Any, Unit] = { case y: Int => ??? case ComplexThing(x, y, z) => ??? }  Here you could do something like test isDefinedAt 1 to check for acceptance of that value, however, what I really want to do is check for acceptance of all Ints (more specifically, in my case the type I want to check is awkward to initialize (it has a lot of dependencies), so I would really like to avoid creating an instance if possible - for the moment I'm just using nulls, which feels ugly). Unfortunately, there is no test.isDefinedAt[Int]. I'm not worried about it only accepting some instances of that type - I would just like to know if it's completely impossible that type is accepted. Portland Pattern Repository Planets Quarks Kuaiper Electron Magma Satellites (by dsl-64-56-227-32.tor.primus.ca 32 hours ago) UnixOverflow sslserver using 100% cpu I've a qmail setup with sslserver being used to secure the pop connections. Normally it runs very well, however some instances of sslserver start taking 100% cpu and its child process qmail-pop3d process is seen as zombie. The process has to be killed to solve the prb. The sslserver version is 0.70. OS is CentOs 6.0 64 bit with 2.6.32-220 kernel. Any clues how I can debug this ? StackOverflow Installing pymatbridge on Windows In version 0.3 of pymatbridge, an additional dependency was added for ZMQ. To install this version, you might therefore need to compile a messenger mex extension linking against a dll of zeromq. However, this turns out to be rather difficult to do in some cases. Does anyone have examples of success stories installing pymatbridge from source on a machine running Windows? Do you need to install Scala separately if you use sbt? Reason I ask, is because it's possible to specify a Scala version in the build.sbt file (using scalaVersion setting), and once you do that, sbt will automatically download that Scala version to use with the project. I also seem to remember that despite having Scala 2.11.1 on my system, sbt would compile and run with Scala 2.10 if no version was specified. So the question is, do I need to install Scala separately if I got sbt installed? Future with State monad I would like to use State monad to implement caching for data provided from third party API. Let's imagine method getThirdPartyData(key: String) which firstly checks cache then if it's not present there should make request to API. First and most naive implementation which came to my mind was enclosing State type within Future - Future[State[Cache, ThirdPartyData]]  But it's not correct because when request fails you will lose your cache (getThirdPartyData will return Failure). Second option which came to my mind was to extend, or rather redefine State monad - s => Future[(s,a)], instead of s => (s,a), but I thought that it's quite popular problem so scalaz probably has some already defined way to solve this issue. Any help greatly appreciated! Clojure - how to use my personal debug library in multiple projects? I have a mascip.debug library with a few functions that I like to use everywhere. In each of my projects (only two at the moment - I'm a new Clojurian), I have this library in src/. I would like to put it somewhere in my PATH, and then be able to use it in any project that I write. Is that silly? Possible? Or would you say I absolutely should start using Maven, as explained here, so I can declare the dependency in my ~/.lein/profiles.clj file? QuantOverflow Comparison of multicurve calibration methods It seems that there are mayor softwares around offering a multicurve framework based on bootstrap. I find this puzzling nowadays, given the distinct advantages of best-fit optimization methods and the hurdles in extending bootstrap techniques to the multicurve setting (e.g. cyclic interdependencies among curves, nontrivial products, usage of nonliquid instruments, overall coherence, dates mismatches, TOY effect, pre-first-tenor forwards, joint curves+term structure dynamics calibration etc). Therefore I must be missing some major drawback of full calibration or overestimating issues for bootstrap. The literature is outdated and mostly partisan, just like those I asked to dismiss either choice altogether. Could someone please shed some more light on these two possibilities and respective drawbacks? (Moreover, Henrard mantains that for best-fit calibration a Newton-Rhapson optimizer suffices, while in my opinion the landscape is not so well-behaved... any views on this?) StackOverflow How to disable http port in play framework? When I run Play Framework using https port, application starts on two ports - default (9000) and https (443). How to disabled default 9000 port and run the Play application ONLY on https port? I run application through the following command: play -Dhttps.port=443 -Dhttps.keyStore=/path/to/keystore -Dhttps.keyStorePassword=password start  I get some logs: [info] play - Application started (Prod) [info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000 [info] play - Listening for HTTPS on port /0:0:0:0:0:0:0:0:443 /r/clojure A Case for Luminus StackOverflow Scala + Gradle + AndroidStudio 0.8.+ Is it still impossible to write Android applications with Scala/Gradle/AndroidStudio pack? I couldn't find anything relevant on the web. Links would be great. Thanks. Fefe US-Außenminister Kerry ist nicht beeindruckt von der ... Asthmatiker stirbt, nachdem das NYPD ihn in den Schwitzkasten ... Wusstet ihr eigentlich, dass Apples Tablets und Mobiltelefone ... Wusstet ihr eigentlich, dass Apples Tablets und Mobiltelefone mit einem Paket-Sniffer kommen? Der aus der Ferne aktiviert werden kann? Der Iran hat die Hälfte seines Bestands an hochangereichertem ... Der Iran hat die Hälfte seines Bestands an hochangereichertem Uran unschädlich gemacht. Für Atomkraftwerke braucht man 5% Anreicherung, für Kernwaffen 90%, der Iran hatte Uran mit 20%. Schade nur, dass Israel gerade eine Gaza-Offensive fährt, so fällt diese Meldung komplett unter den Tisch. StackOverflow Why is Scala's syntax for tuples so unusual? In mathematics and computer science, a tuple is an ordered list of elements. In set theory, an (ordered) n-tuple is a sequence (or ordered list) of n elements, where n is a positive integer. So, for example, in Python the 2nd item of a tuple would be accessed via t[1]. In Scala, access is only possible via strange names t._2. So the question is, why can't I access data in tuples as Sequence or List if it is by definition? Is there some sort of idea or just yet not inspected? Eclipse: How do I quickly open a folder (in the package, project or resource explorer) with a keyboard shortcut? I am used to quickly search and open files with CTRL+SHIFT+R - Open Resource. However, Open Resource does not match folders. I would need to do the same for folders within the project, or alternatively, for matching files with part of their project-relative path. The use case is working with Ansible roles, where all files are named "main.yml" and are located in a hierarchical folder structure: -project +-roles +-apache | +---tasks | +----main.yml +---handlers | +----main.yml +---vars | +----main.yml  And I need to quickly open the various main.yml files by searching for their path e.g. typing something like CTRL + SHIFT + <<insert key here>>  then apache/vars  and obtain a short list including apache/vars/main.yml Is there any feature or plugin providing such functionality? It would also be useful for opening maven pom.xml files and the like... Scala : statements are not executed in order I am learning scala I tried following program package com.avdongre object MyModule { def abs(n: Int): Int = if (n < 0) -n else n private def formatAbs(x: Int) = { val msg = "The absolute value of %d is %d" msg.format(x, abs(x)) } def factorial(n: Int): Int = { def go(n: Int, acc: Int): Int = if (n <= 0) acc else go(n - 1, n * acc) go(n, 1) } def main(args: Array[String]): Unit = println(formatAbs(-42)) println(factorial(5)) }  I get following output 120 The absolute value of -42 is 42  Why factorial is getting called first ? Java / Scala import not working due to incorrect build path? I'm very confused about how you import packages and their classes. I have two scala projects with the following directory structures: project1/ src/ main/ scala/ utils/ some_file.scala worksheet/ learning.sc project2/ * seemingly the same as project 1 *  The contents of some_file.scala: package utils class InterestingClass { override def toString(): String = "I should be interesting" }  The contents of learning.sc: package worksheet import utils.InterestingClass object learning { println("Welcome to the Scala worksheet") //> Welcome to the Scala worksheet println(new InterestingClass()) //> I should be interesting }  When I created project1/src/main/scala/, in Eclipse I right clicked main.scala that was showing as a package underneath src and selected: Build Path > Use As Source Folder, and in Eclipse then "deleted" the src folder. project2 is actually a project I have downloaded as part of an online course so I guess there's some config in the .project file or the .settings folder that I need to tweak because learning.sc in project1 auto compiles as expected and produces the two outputs including I should be interesting but in project2 it does not and the line import utils.InterestingClass shows the error not found: object utils. I guess there's a subtle different in the build paths but I can't find it and the error message doesn't get me much further. Any tips on how to debug import errors? Looking at the build paths, they look correct (both have project1/src/main/scala or project2/src/main/scala). Planet Emacsen Irreal: Backup Fonts for Unicode Glyphs Trey Harris posed a question on the Google+ Emacs Community that’s been bothering me for some time. I use the terrific Inconsolata as the default for monospaced fonts but many of the Unicode glyphs are missing. Mac OS X takes care of finding missing glyphs in native apps like Terminal.app or Mail.app but with Emacs I just get a missing glyph. I’ve tried various fixes over the years but nothing worked very well. Harris suggested using the unicode-fonts package to handle this. Happily, it’s an ELPA package so I just loaded it up and added the required lines to my init.el and those missing glyphs starting appearing in Emacs. Years of frustration gone just like that. There are still a few missing characters but the README for unicode-fonts lists a few font packages that you can load to supply most characters you’re apt to need. Even without loading the extra fonts, I’ve got almost everything I need so I haven’t bothered with them. If you’ve been having a problem with missing Unicode glyphs, give unicode-fonts a try. It’s easy to load with ELPA and if it doesn’t work for you, it’s easy to uninstall. It’s a huge win for me. CompsciOverflow Are all Chomsky-Type3 grammars LL(1)? Referring to this Question, where an answer is stating that all Type 3 languages are LL(1), I'd like to know if all Type 3 grammars are possibly LL(1). If not, why is it so? Are there maybe ambiguous grammars in the set of the Type 3 grammars? TheoryOverflow Complexity of an algorithm for deciding 3-colorability of graph by the chromatic polynomial modulo x-3 As explained on MO computing the chromatic polynomial P(G,x) modulo x-3 is enough for deciding 3-colorability. For non adjacent vertices u and v, G+uv is the graph with the edge uv added and the graph G/uv is obtained by merging the two vertices. According to wikipedia$$P(G,k)= P(G+uv, k) + P(G/uv,k) \qquad (1)$$The recursion for computing$P(G,x)$modulo$x-3$is siginificantly faster since induced$K_4$stops computing further. Consider the following naive algorithm for checking 3-colorability using$P(G,x) \pmod{x-3}$.  function color3(G: connected graph) #returns 0 if G is not 3-colorable, otherwise reports it is and stops if G contains induced K_4 return 0 if G is a clique on 3 or less vertices report '3-colorable'. Stop. if we can add edge uv which induces K_4 in G return color3(G/uv) #the induced K_4 contributes 0 to the sum #the above makes the problem strictly smaller #this might be greatly improved by trying to induce K_4-e [C] pick the lexicographic first non-adjacent u,v and return color3(G+uv)+color3(G/uv)  One call to color3 is polynomial, so the complexity depends on the number of double recursion in [C]. According to Wikipedia the worst case for computing (1) is$\phi^{n+m}$. 3-coloring 4-regular graphs is NP-complete, so for 4-regular it is$\phi^{3n}$. Appears to me terminating by induced$K_4$would greatly improve the running time. Questions: Q1 What is the complexity of color3?$O(c^n)$for small$c$will be of practical interest.$2^{o(n)}$might indicate complexity collapse. Q2 How to improve [C]? There might be other improvements like other 4-chromatic subgraph or$Fbeing in a graph class where coloring is polynomial. Experimental results: For several 4-regular graphs on 246 vertices which are the line graphs of 3-regular graphs color3 found 3-coloring in less than a minute in sage 6.2 implementation. For several smaller graphs which are not 3-colorable it correctly returned 0. For reduction SAT to 3-color the running time was fast too. Sage implementation: https://gist.github.com/jor0/039127ab69dd8934c105 StackOverflow Sending and receiving different types of messages with spray I am using spray routing and spray client. I see that pipleline in spray client allows to define, by default, one and only one type of answer. I want to expose with spray routing a rest json service which replies either with a ServiceResponse type, either with a Error response type. I can define types and marshalling of ServiceResponse and Error on spray routing, and define a route which returns either a ServiceResponse or an Error. I don't see how to call that from spray client, since the result would be a Future[ServiceResponse] or Future[Error]. The only solution I made work is to define an encapsulation : MetaResponse(Option(ServiceResponse), Option[Error]). In the spray client I use pattern matching to know in which case I am. But I find this solution ugly. Is there a way to specify that a given client request can expect several return types? Thanks. /r/compsci Please try to break my login system... Hi everyone! As a test for a project I just came up with a login system which I'm hoping is secure without needing to use TLS/SSL. It works in a similar way to HTTP digest authentication, CRAM-MD5, etc., but only uses JS and Crypto-JS (for SHA512) on the client side. Now before you ask, no this is not meant to be a replacement for TLS. Of course TLS is far better and this system is simply designed to stop packet sniffers and the like from stealing passwords in plain text. It doesn't stop session hijacking and it doesn't encrypt normal traffic, it just secures the login (hopefully). I have included some example data that would have been captured by packet sniffing during a valid login for the "admin" user. My question is, can anybody use the form and the data given to find admin's password? Also if anybody finds any flaws in the system I'd love to hear about them. Thanks all, I'll look forward to hearing your comments! submitted by polaris64 [link] [2 comments] StackOverflow Functional way of joining two js object arrays based on common id I am trying to acheive something similar to SQL table join, in the most elegant (functional) way, preferably with underscore.js, so no for loops please. I need to merge objects from two different arrays, matched upon a common identifier. For example, given: var basic = [{ id: '1', name: 'someName', }, {...} ] var ext= [{ id: '1', job: 'someJob', }, {...} ]  Result should be: var combined = [{ id: '1', name: 'someName', job: 'someJob', }, {...} ]  Thanks! Parallelize Scala's Iterator Please note: This is not a duplicate question, since this question specifies on all methods Iterator has, not just map and flatMap. Therefore Future.traverse are not a good answer. Let's say I have this simple statement: (1 to 100).toSet.subsets.find(f)  It works perfectly. It's lazy, doesn't use a lot of memory and returns as soon as one element is found. The problem starts when you want to parallelize it. You might say, it's Scala, there has to be .par or Iterator, but there isn't. The proposed solution on the internet is to use .grouped, but it's not as good as I'd want. Why? val it = (1 to 100).toSet.subsets.grouped(1000000).map(_.par.find(f)).flatten if (it.hasNext) Some(it.next) else None  1. Uses much more memory. I know it's still O(1), but let's be perfect here :) 2. It's not perfectly parallelizable (by Amdahl's law). When .grouped is consuming the iterator for next block of million elements, all but one thread waits. This is especially problematic if iterator is expensive to consume. Moreover, there has to be an overhead of spawning a new set of threads to work on a new block. 3. Produces more complicated / longer code (see example). It would shorten code a bit if Iterator had .nextOption, but still. Is there anything else despite programming my own producer-consumer model (iterator is producer, threads are consumers) and then final reduce step? How to remove an item from a list in Scala having only its index? I have a list as follows: val internalIdList: List[Int] = List()  internalIdList = (11, 12, 13, 14, 15) From this list would remove the third element in order to obtain: internalIdList = (11, 12, 14, 15) I can not use a ListBuffer, are obliged to maintain the existing structure. How can I do? Thanks to all Fefe Lustiger Vortrag von djb. Er macht sich da Gedanken, ... Lustiger Vortrag von djb. Er macht sich da Gedanken, wie man, wenn man die NSA ist, dafür sorgen könnte, dass die Software da draußen schön unsicher bleibt. Money Quote (aus dem Cybersecurity-Blablah der US-Regierung): Subcategories in Framework: 98. … promoting secure software: 0. This is how the money is spent. Inhaltlich hat er natürlich völlig Recht. Das ganze System baut darauf auf, dass alle Beteiligten mit Bullshit abgelenkt werden, damit es nicht dazu kommt, dass jemand tatsächlich was tut, um die Lage zu verbessern. CompsciOverflow Convert DFA to Regular Expression In this old exam-task I don't understand all the steps to convert the DFA below to a Regular Expression. The q_2 state is eliminated first. The provided solution to eliminate q2 is: If we first eliminate q2 we obtain an intermediate automata with 3 states (q0 , q1 and q3 ) such that: 1. We go from q0 to q1 with RE a+ba 2. We go from q0 to q3 with RE bb 3. We go from q1 to q1 with RE ba 4. We go from q1 to q3 with RE a+bb I don't understand nr2. q3 can also be reached using RE aa. Why is this left out? : : StackOverflow Scala framework for a Rest API Server? We are thinking on moving our Rest API Server (it is inside the web service, on Symfony PHP) to Scala for several reasons: speed, no overhead, less CPU, less code, scalability, etc. I didn't know Scala until several days ago but I've been enjoying what I've been learning these days with the Scala book and all the blog posts and questions (it's not so ugly!) I have the following options: • build the Rest API Server from scratch • use a tiny Scala web framework like Scalatra • use Lift Some things that I will have to use: HTTP requests, JSON output, MySQL (data), OAuth, Memcache (cache), Logs, File uploads, Stats (maybe Redis). What would you recommend? What are alternatives to JDBC driver for access PostgreSQL database I'm using official JDBC driver for PostgreSQL, but I'm stuck with the following issues: • No support for PostgreSQL-ish data structures such as UUIDs. • Common JDBC weirdness, such as: • No function to escape values for consuming by PostgreSQL. • Limited support for executing heterogeneous statements in batch. • No rewriting of multiple insert statements into single insert statement when inserting many rows in one table. So, the question — is there any PostgreSQL database driver which can leverage full power of PostgreSQL without much boilerplate? I'm also use Scala language for development, so if driver is designed specifically for Scala it would be so much awesome awesome. QuantOverflow Worked examples of applying Ito's lemma In most textbooks Ito's lemma is derived (on different levels of technicality depending on the intended audience) and then only the classic examples of Geometric Brownian motion and the Black-Scholes equation are given. My question I am looking for references where lots of worked examples of applying Ito's lemma are given in an easy to follow, step by step fashion. Also more advanced cases should be covered. Undeadly Hibernating to Encrypted softraid(4) Now Supported With this commit, Mike Larkin (mlarkin@) has added support for hibernating to encrypted softraid(4) devices. This is what he had to say when asked about it: After RLE support (which went in in Slovenia), the next thing on the list to tackle was softraid crypto. Theo provided the initial idea on how to get the block transforms and crypto bits working over lunch one day in Slovenia and after about three or four days of on-and-off hacking this week, we had it working. Read more... Fefe Das ZDF spricht eine Reisewarnung für Österreich ... Das ZDF spricht eine Reisewarnung für Österreich aus, empfiehlt "ausschließlich unpolitische Aktivitäten zu entfalten" . Fnord des Tages: "FDP-Mitglied wegen CDU-Zusammenarbeit ... StackOverflow Apache Spark:Which one is more efficient? Code is writed by scala,I'm wonder which one is more efficient:  src.filter(word=>!word.contain("xx").map{word=>(word,1))  or :  //src:Array("xx","c","d") //if word contains xx,fuc will return (),how to deal with it ? src.map(word=>if(!word.contain("xx")(word,1)) //result:Array((), (c,1), (d,1))  If word contains xx,fuc will return (),how to deal with it ? What is the inverse of intercalate, and how to implement it? This question discusses how to interleave two lists in an alternating fashion, i.e. intercalate them. • What is the inverse of "intercalate" called? • Is there an idiomatic way to implement this in Scala? Undeadly g2k14: Stefan Sperling on wireless drivers I spent most of this hackathon looking at problems in wifi drivers. I wasn't exactly sure in advance which problems I wanted to work on. So I packed a bunch of hardware, including several USB wifi adapters, (rsu(4), 2x run(4), rum(4), urtwn(4), zyd(4)), some miniPCIe cards (an unsupported cousin of urtwn(4) named Realtek 8188CE, unsupported athn(4) AR9485, bwi(4)), two laptops, and an access point. This left me with more than enough toys for a week. Read more... g2k14: Florian Obser in IPv6 land I arrived in Ljubljana somewhat tired so I started the first day off with some light ping(8) and ping6(8) hacking. Some unifdef(1) application for #ifdef FEATURE_THAT_EXISTS_SINCE_FOREVER_BUT_MAYBE_WE_DONT_HAVE_IT and some cleanup by hand. The idea is to have ping(8) and ping6(8) be the same binary like traceroute(8) and traceroute6(8). Read more... QuantOverflow Does Modern Portfolio Theory align with EMH? I came to the conclusion that in literature Markowitz' Portfolio Theory is believed to be compliant with the Efficient Market Hypothesis. The weakest form states that the current price fully incorporates information contained in the past history of prices only. That is, nobody can detect mispriced securities and “beat” the market by analyzing past prices. I understand this means that future price movements are independent from what happened until now. In MPT though, to allocate funds between assets, we estimate the expected return and covariances from past prices. If we want to be compliant with EMH, shouldn't we believe that those estimates make no sense? StackOverflow Use '==' instead of equals for string comparison in Scala Why is it recommended to use == rather than .equals for string comparison in Scala? There are a lot of questions in StackOverflow that do not recommend the use of reference equality for String comparison in Java but why is it advised to do the exact reverse in Scala? SBT 0.13 taskKey macro doesn't work with [Unit]? lazy val buildDb = taskKey[Unit]("Initializes the database") buildDb := { (compile in Compile).value val s: TaskStreams = streams.value s.log.info("Building database") try { ... } catch { case e: Throwable => sys.error("Failed to initialize the database: " + e.getMessage) } s.log.info("Finished building database") }  This produces the following error C:\work\server\build.sbt:98: error: type mismatch; found : Unit required: T s.log.info("Finished building database") ^ [error] Type error in expression  But if I define it like this lazy val buildDb = taskKey[String]("Initializes the database") and then add to the last line in the task "Happy end!" string everything seem to work. Am I to blame, or something wrong with the macro? Undeadly g2k14: Ingo Schwarze on manly stuff In the week right before the hackathon, I have done quite a bit of work cleaning up mandoc(1) warning and error messages. The goal is to provide more, more precise, and more readily understandable information to the user, in particular mentioning in the messages which section titles, macro names, and arguments each individual message is related to, and which workaround or fallback mandoc(1) has chosen, if any. Read more... StackOverflow Sort list of string with localization in scala i want to sort a list of strings. I know that's not difficult in scala but my problem is, that i need to sort lists in different languages. For example i know that i can sort strings in english very easily. But what's about the russian language or the romanian one? What is the best practice for sorting strings in multiple languages in scala? Does the scala sorting implementation support only english letters? In java i would do something like this: Collator coll = Collator.getInstance(locale); coll.setStrength(Collator.PRIMARY) Collections.sort(words, coll);  I hope someone out there can help me. Thanks in advance Nico. Fefe SMS des Tages. Lobsters Introduction to A* DataTau Introduction to Markov chain : simplified! StackOverflow ansible : how to pass multiple commands I tried this : - command: ./configure chdir=/src/package/ - command: /usr/bin/make chdir=/src/package/ - command: /usr/bin/make install chdir=/src/package/  which works but I guess there is something more.. neat. So I tried this : from : multiple commands in the same line for bruker topspin which give me back "no such file or directory" - command: ./configure;/usr/bin/make;/usr/bin/make install chdir=/src/package/  but I couldn't find the right syntax to put : - command: "{{ item }}" chdir=/src/package/ with_items: ./configure /usr/bin/make /usr/bin/make install  That does not work, saying there is a quote issue. Anyone ? Thank you /r/compsci Share your stories about algorithms in action, in real-life. Have a personal algorithm that you use on a daily basis? Own an electrical appliance with smart behavior? Noticed algorithms in action, in real life? Let's share them. Here's an interesting "optimization" algorithm I've noticed in my living area. I live in a residential block that is 25-stories high. The 2 elevators/lifts (US/UK designation respectively) used to be slow moving machines that caused huge waiting times during morning rush hours when people were leaving home for work. So recently, the elevators got an uplift. Besides being mechanically upgraded, it seems that their software was made intelligent. 1) If there is exactly one idle elevator (A) while B is moving upwards, A will move to the ground floor to wait. 2) If there is exactly one idle elevator (A) while B is moving down towards the ground floor, A will move to the 17th floor to wait. 3) If both elevators are idle, one will be randomly chosen to move to ground floor, and the other to the 17th floor to wait. A and B are inter-changeable. What is this achieving? Firstly, the lift engineers have recognized that the ground floor is the most accessed floor, so having one elevator wait there at all times will likely minimize the sum of waiting time throughout the day. Then, what's so special about the 17th floor? Turns out that if you have idle elevators at the ground (1st) and 17th floors, the maximum amount of floors that either elevator has to traverse to reach a passenger anywhere is minimized at 8 (minimax strategy). For example, the elevator at the ground floor can reach floors 2-9 within 8 floors, while the elevator at the 17th floor can reach floors 9-25 within 8 floors. We no longer have two derpy elevators that idly wait at the 25th floor, in the worst case. Simple algorithm that has made my life better! submitted by crimsonSoldier [link] [41 comments] /r/netsec Havex, It’s Down With OPC CompsciOverflow Which textbook can I use after a high school CS course? I just finished the AP Computer Science course in high school and since my school does not have any further classes, I was thinking about getting a textbook and continuing my study. APCS goes through Big O, Arrays + ArrayLists, Recursion, Linked Lists, a rough overview of some other data structures and some sorting and searching algorithms (all in Java). I will start Calculus in school next year (if that helps to determine math level). Since I was thinking about participating in CS tournaments, I thought it would be useful to learn more about algorithms and data structures. I was looking at two textbooks, The Algorithm Design Manual by Steven S Skiena and Introduction to Algorithms by Cormen et. al. I was wondering which one, if any of these, would best fit my situation. StackOverflow How to apply new changes without dropping current data in the db.(Playframework evaluations) I'm using play 2.2.1 with scala. And I have a managing database evaluations. When I run my application with changes on the database, it'll drop all my datas. At this moment evaluationplugin = disabled is comment in. If I comment out, it does not apply my changes. For example. I have a users table and there are id, f_name, l_name User id f_name l_name 1. khazo rasp  And I want to add age field to this table without losing data. I've added this field in scala files.It works properly. I'm assuming I need to write script in 1.sql for some alter command, but I don't want to write script. How to apply new changes without dropping current data in the db. I've read this documentation. Thanks in advance. Fred Wilson Free International Roaming With A Premium Upsell I just landed in Berlin after an overnight flight from the US. In the past, turning on your phone after landing overseas could be an expensive experience as the phone downloads all the email you received since taking off at international mobile data rates. I’ve used a host of techniques over the years to avoid the experience of landing, turning on my phone, and immediately getting a text message that I’ve blown past my international data roaming cap. I’ve turned off mobile data and waited until I got to hotel WiFi to download my email but that meant no mobile data for directions to the hotel. I’ve bought SIM cards in airports. And more recently I’ve rented a pocket WiFi before traveling overseas. But last year the Gotham Gal and I switched back to T-Mobile after they introduced free low bandwidth international data roaming for all customers in the US. Here is the experience when I land. I turn on the phone, it finds the local mobile network, connects, and my phone lights up with notifications and emails start coming in. In addition I get a text message from T-Mobile offering to upgrade me to an international data pass that offers 4G in 100MB buckets at roughly10/100MB.

I buy the upgrade every time and am happy to pay for the higher speeds.

But the important thing here is the customer experience. No longer do customers have to fear turning on their phone. No longer do customers have to jump through hoops to procure an affordable mobile data plan. If you want faster speeds, T-Mobile makes it drop dead simple to upgrade right on your phone.

This approach to international mobile data should be adopted by all the mobile carriers. It’s a great experience.

Planet Clojure

Machine Learning in Clojure - part 2

I am trying to implement the material from the Machine Learning course on Coursera in Clojure.

My last post was about doing linear regression with 1 variable. This post will show that the same process works for multiple variables, and then explain why we represent the problem with matrices.

The only code in this post is calling the functions introduced in the last one. I also use the same examples, so post this will make a lot more sense if you read that one first.

For reference, here is the linear regression function:

(defn linear-regression [x Y a i]  (let [m (first (cl/size Y))        X (add-ones x)]    (loop [Theta (cl/zeros 1 (second (cl/size X))) i i]      (if (zero? i)        Theta        (let [ans (cl/* X (cl/t Theta))              diffs (cl/- ans Y)              dx (cl/* (cl/t diffs) X)              adjust-x (cl/* dx (/ a m))]          (recur (cl/- Theta adjust-x)                   (dec i)))))))
Because the regression function works with matrices, it does not need any changes to run a regression over multiple variables.

Some Examples

In the English Premier League, a team gets 3 points for a win, and 1 point for a draw. Trying to find a relationship between wins and points gets close to the answer.

(->> (get-matrices [:win] :pts)    reg-epl    (print-results "wins->points"))** wins->points ** A 1x2 matrix ------------- 1.24e+01  2.82e+00
When we add a second variable, the number of draws, we get close enough to ascribe the difference to rounding error.

(->> (get-matrices [:win :draw] :pts)     reg-epl     (print-results "wins+draws->points"))** wins+draws->points ** A 1x3 matrix --------------2.72e-01  3.01e+00  1.01e+00
In the last post, I asserted that scoring goals was the key to success in soccer.

(->> (get-matrices [:for] :pts)     reg-epl     (print-results "for->points"))** for->points ** A 1x2 matrix ------------- 2.73e+00  9.81e-01
If you saw Costa Rica in the World Cup, you know that defense counts for a lot too. Looking at both goals for and against can give a broader picture.

(->> (get-matrices [:for :against] :pts)     reg-epl     (print-results "for-against->pts"))** for-against->pts ** A 1x3 matrix ------------- 3.83e+01  7.66e-01 -4.97e-01
The league tables contain 20 fields of data, and the code works for any number of variables. Will adding more features (variables) make for a better model?

We can expand the model to include whether the goals were scored at home or away.

(->> (get-matrices [:for-h :for-a :against-h :against-a] :pts)     reg-epl     (print-results "forh-fora-againsth-againsta->pts")) ** forh-fora-againsth-againsta->pts ** A 1x5 matrix ------------- 3.81e+01  7.22e-01  8.26e-01 -5.99e-01 -4.17e-01
The statistical relationship we have found suggests that that goals scored on the road are with .1 points more than those scored at home. The difference in goals allowed is even greater; they cost .6 points at home and only .4 on the road.

Wins and draws are worth the same number of points, no matter where the game takes place, so what is going on?

In many sports there is a “home field advantage”, and this is certainly true in soccer. A team that is strong on the road is probably a really strong team, so the relationship we have found may indeed be accurate.

Adding more features indiscriminately can lead to confusion.

(->> (get-matrices [:for :against :played :gd :for-h :for-a] :pts)     reg-epl     (map *)     (print-results "kitchen sink”))** kitchen sink **(0.03515239958218979 0.17500425607459014 -0.22696465757628984 1.3357911841232217 0.4019689136508527 0.014497060396707949 0.1605071956778842)
When I printed out this result the first time, the parameter representing the number of games played displayed as a decimal point with no digit before or after. Multiplying each term by 1 got the numbers to appear. Weird.

The :gd stands for “goal difference” it is the difference between the number of goals that a team scores and the number they give up. Because we are also pulling for and against, this is a redundant piece of information. Pulling home and away goals for makes the combined goals-for column redundant as well.

All of the teams in the sample played the same number of games, so that variable should not have influenced the model. Looking at the values, our model says that playing a game is worth 1.3 points, and this is more important than all of the other factors combined. Adding that piece of data removed information.

Let’s look at one more model with redundant data. Lets look at goals for, against and the goal difference, which is just the difference of the two.

(->> (get-matrices [:for :against :gd] :pts)     reg-epl     (print-results "for-against-gd->pts"))** for-against-gd->pts ** A 1x4 matrix ------------- 3.83e+01  3.45e-01 -7.57e-02  4.21e-01
points = 38.3 + 0.345 * goals-for - 0.0757 * goals-against + 0.421 * goal-difference

The first term, Theta[0] is right around 38. If a team neither scores nor allows any goals during a season, they will draw all of their matches, earning 38 points. I didn’t notice that the leading term was 38 in all of the cases that included both goals for and against until I wrote this model without the exponents.

Is this model better or worse than the one that looks at goals for and goals against, without goal difference. I can’t decide.

Why Matrices?

Each of our training examples have a series of X values, and one corresponding Y value. Our dataset contains 380 examples (20 teams * 19 seasons).
Our process is to make a guess as to the proper value for each parameter to multiply the X values by and compare the results in each case to the Y value. We use the differences between the product of our guesses, and the real life values to improve our guesses.

This could be done with a loop. With m examples and n features we could do something like

for i = 1 to m      guess = 0      for j = 1 to n           guess = guess + X[i, j] * Theta[j]      end for j difference[i] = guess - Yend for i
We would need another loop to calculate the new values for Theta.

Matrices have operations defined that replace the above loops. When we multiply the X matrix by the Theta vector, for each row of X, we multiply each element by the corresponding element in Theta, and add the products together to get the first element of the result.

Matrix subtraction requires two matrices that are the same size. The result of subtraction is a new matrix that is the same size, where each element is the difference of the corresponding elements in the original matrices.

Using these two operations, we can replace the loops above with

Guess = X * Theta
Difference = Guess - Y

Clearly the notation is shorter. The other advantage is that there are matrix libraries that are able to do these operations much more efficiently than can be done with loops.

There are two more operations that our needed in the linear regression calculations. One is multiplying matrices by a single number, called a scalar. When multiplying a matrix by a number, multiply each element by that number. [1 2 3] * 3 = [3 6 9].

The other operation we perform is called a transpose. Transposing a matrix turns all of its rows into columns, and its columns into rows. In our examples, the size of X is m by n, and the size of Theta is 1 x n. We don’t have any way to multiply an m by n matrix and a 1 by n matrix, but we can multiply a m by n matrix and an n by 1 matrix. The product will be an m by 1 matrix.

In the regression function there are a couple of transposes to make the dimensions line up. That is the meaning of the cl/t expression. cl is an alias for the Clatrix matrix library.

Even though we replaced a couple of calculations that could have been done in loops with matrix calculations, we are still performing these calculations in a series of iterations. There is a technique for calculating linear regression without the iterative process called Normal Equation.

I am not going to discuss normal equation for two reasons. First, I don’t understand the mathematics. Second the process we use, Gradient Descent, can be used with other types of machine learning techniques, and normal equation cannot.

Planet Clojure

Clojure Gazette 1.85

Clojure Gazette 1.85
 Magic, Games, Shootout

Issue 1.85 July 20, 2014

Editorial

Yes, it's true. I have the best newsletter readers on the planet. You all are awesome and I'm continually blown away with the support you show me each time I ask. :)

And now, I ask for something again ;)

I've been writing the Clojure Gazette for over two years now. It's brought me joy. It's brought me pain. But I keep at it and I have grown to love it. It started on the free plan of the newsletter service but graduated some time ago to the not-free plan. I've paid for everything myself and continued to build the list.

Although the growth has been steady in the last year, the slope was way lower that I would have liked. I put any marketing efforts on the back burner, not really worrying about it too much. But now, I can't ignore it.

I have recently reworked the Clojure Gazette landing page to be a little bit snazzier and I'm starting to link to it more regularly from my blog. But I need your help.

There are two ways you can help: one is to send me a testimonial. Just a short blurb about how your life is better because you receive the Gazette and a profile-style picture. Be honest! Why do you read the Gazette? I can't promise I'll put yours on the landing page, but I will send you a nice thank you. A lot of people have sent me great emails over the years, but I never asked permission to publish them so I don't feel right putting them on the page.

The second way to help is by sharing it with your friends. It came as a surprise to me, but some people have never heard of the Clojure Gazette! If you know one of these people, it is your duty to inform them of its existence ;) And if you have any kind of internet presence (blog, Twitter, podcast, etc), I would greatly appreciate links, tweets, likes, reblogs, shares, plugs, +1s, hearts, favorites, thumbs up, high fives, pins, retweets, blogs, posts, or whatever your respective social networking technology uses. In return, I will issue fives, high fives, and low fives (unless you're too slow) to any and all at events that I frequent.

Sincerely,
Eric Normand <ericwnormand@gmail.com>

Of Mages and Grimoires

Reid McKenzie has created new Clojure documentation site called Grimoire and this post discusses the reasoning behind it. It looks like a cool project to make the documentation very useful for quick access (faster than ClojureDocs). It also has lots of examples and the collection is growing. I'm glad someone is working on this. Check out, as an example, the page for defprotocol. It includes, in one place, the docstring, examples, and source. I'd love to see hyperlinks in the code to their respective definitions. DISCUSS

Clojure web server shootout

Charts and graphs comparing the performance of different Clojure web server options. I'm surprised to see Jetty Ring fare so poorly. A great service to the community. It's a collaborative repo and they are accepting benchmarks. DISCUSS

Brute - Entity Component System for Clojure

I have a confession to make: ever since I was young, I've wanted to program games. So, far from bein an unbiased link curator, I do have a softspot for game-related programming posts. This one is an Entity Component System, which is a pattern for composing game objects instead of writing them from scratch each time or using inheritance. And now it works in ClojureScriptDISCUSS

Room Key Case Study

I almost didn't link to this because it was not very technical, but then I thought that it was such a good document on so many other levels that it was worth sharing. It is well designed, well written, and well scoped. It would make a great post to email to a project manager to convince them to give Clojure and Datomic a try. DISCUSS

ki

If game programming was one of my first programming loves, programming languages have to be second. And that love has been much more requited :) ki is an interesting language: it's a sweet.js macro that expands to Javascript, but inside the macro, it's a Lisp. The compiler can run in your browser. It uses mori for its persistent data structures. Very interesting! DISCUSS

Building Single Page Apps with Reagent

Om is all-the-rage, but there are alternative React libraries out there. Reagent is one of them and this post begins with a discussion of the advantages of Reagent over Om, then goes on to show the code and what kinds of things Reagent is good at. DISCUSS

Elm – functional reactive dreams + missile command

A smart, in-depth look at Elm, which is a great new language that's worth watching. DISCUSS

nginx-clojure

A Ring adapter embedded in nginx? Wowee, that's fast! (It's currently winning the Clojure web server shootout.) DISCUSS

Wes Felter

Sandstorm.io Personal Cloud Platform Crowdfunding

Sandstorm.io Personal Cloud Platform Crowdfunding:

I’ve been going on about the personal cloud on this blog for years. Sandstorm is the first project that I would back with my own money. They’re discarding all the user-hostile Linux baggage to create a platform that just works.

Planet Clojure

Recommended Reading : Clojure for the Brave and the True ( for Beginners)

If you are learning Clojure, one of the hottest languages, there are lots of good books, web sites and more.

A relatively new resource from braveclojure.com :

TheoryOverflow

How to measure programming language succinctness?

I want to explore the notion of quantifying the amount of succinctness a programming language provides. That is, the amount a high-level language reduces the complex.

This idea of "simplification" is a factor of text-wise reduction (fewer characters needed to express a complex concept, à la Algorithmic Information Theory) and another, less easy-to-quantify concept of maintainability. Fleshing out this latter concept, it is clear it has to do with how easily one can establish programmer consensus for the given task (i.e. how many programmers of the language would put it back the same way you've expressed it or otherwise agree on the best implementation between different implementations of the same problem?).

I will define the "Kolmogorov Quotient" so that higher numbers for a given language denote a reduction in the complexity of solving the problem in the given language.

The metric for "text-wise reduction" should incorporate a constant factor based on (non-identifier) symbols used in the language and source text. These factors will be the same across all languages implemented (i.e. designed) for a given architecture (e.g. VonNeumann architecture vs. Symbolics) and will be a measure of the significance of the symbol; i.e. the topology of the language tokens. (These terms, alas, will also need to be fleshed out and defined.)

Once the basic premise and a methodology above is agreed to, it is only a matter of a rough constant of difference for any specific implementation/architecture. (That is, as long as the architecture is the same across all measurements, the number should be valid and comparable between languages.)

But it could go something like this: Pick a language "close to the machine", like C or Assembly, and measure the amount of bytes of machine code it used to implement a standard suite(*) of common, non-threaded programming tasks (base_language_count). Then code the exact same functionality in the language you are wanting to measure (without using external libraries) and count the number of bytes of source code (test_language_count).

KQuotient = (base_language_count / test_language_count) / number_of_equal_programs.

"number_of_equal_programs" is the number of programs fluent programmers of the language agree that are the best and equal solutions to the problem. (I will define "equal programs" as those who's output is the same for every input.)

The first ratio should always be greater than 1.0. My only concern is that for each programming language, one could reduce the KQuotient number simply by changing each keyword to a single character.

(*) "standard suite of common programming tasks...": I see two main categories:

1. Data-processing suite, limited to simple text I/O (computation towards the machine)
2. GUI suite (computation towards the user)

The purpose of this idea is to end the tiring "language wars" about whose language is the best. By giving a quantitative metric, people can at least argue better.

/r/emacs

Org Users: What did it take you a long time to understand about org-mode?

I love Org Mode, but I haven't read all or even most of the manual. I just read this passage:

Important: Scheduling an item in Org mode should not be understood in the same way that we understand scheduling a meeting. Setting a date for a meeting is just a simple appointment, you should mark this entry with a simple plain timestamp, to get this item shown on the date where it applies. This is a frequent misunderstanding by Org users. In Org mode, scheduling means setting a date when you want to start working on an action item.

I had previously had this misunderstanding. Now I don't, but I suspect I'm still misunderstanding / under-utilizing other parts of Org. What did you misunderstand or not know about for a long time?

submitted by birdmanx35

StackOverflow

Issue understanding splitting data in Scala using "randomSplit" for Machine Learning purpose

Hi I am new in MLlib and I am reading the documents on Spark website about it. I have difficulty to understand why in the following code we need to cache "0" for training and "1" for testing:

  val splits = data.randomSplit(Array(0.6, 0.4), seed = 11L)
val training = splits(0).cache()
val test = splits(1)


Can anyone help me understanding the reason? As far as I know we need positive and negative samples so "1" can be positive and "0" can be negative, why it is divided like this?

Thank you!

can't get cookie after add cookie to a jsoup session

The following code will print an empty of "{}", why I can't see any cookie. Since I set the cookie in the second line, shouldn't expect return cookie?

var res = Jsoup.connect(url)
.data(postParamsList: _*)
.method(Method.POST)
.execute()


The following will print the cookie after comment out the second line.

var res = Jsoup.connect(url)
.data(postParamsList: _*)
.method(Method.POST)
.execute()


/r/freebsd

State of FreeBSD driver support on mobile hardware?

I'm currently a Slackware user out of the (perhaps misplaced) belief that the BSDs do not have good mobile hardware support, namely wireless and acpi power management. I'm generally fond of the simplicity of FreeBSD and found Slackware to be very similar in the regard, but with, generally, better driver support. Has this changed in recent years? I was reading the thread about the state of graphics support and some of the comments seemed encouraging. If it helps to have a hardware reference, I will be using a Thinkpad x61.

submitted by ewood87

/r/netsec

Just about every web app scanner I've looked at recently, including my beloved BurpSuite, injects references to their own infrastructure (URL, DNS, email, etc) in a way that could leak test and vulnerability data.

I don't necessarily want to call this 'shady' at this point, but i think it's messed up that almost none of these vendors openly disclose the fact that certain tests could leak vulnerability data to their infrastructure unless you take action to prevent it. (Acunetix discloses it with AcuMonitor) The only way to know for sure is to run a full suite of tests to see if you can find any such reference, but given some tests are 'optimized', there's really no way to ensure that you're seeing everything given any particular test scenario.

At this point I run all tests through a proxy that looks for this type of thing and both logs and rewrites it, but it's not always easy to detect this type of thing. It's probably not an issue for 99% of the folks using the product, but it is an issue for me. Not only because of data sensitivity concerns, but sometimes the test flat out isn't going to work because it's an internal site with no outside access. So you wind up with false negatives because the whole thing is tucked away.

Thoughts?

StackOverflow

Proxying sbt ivy repositories with artifactory

I'm trying to do almost exactly what is described in the official sbt documentation, at http://www.scala-sbt.org/0.13.5/docs/Detailed-Topics/Proxy-Repositories.html. I'd like to proxy both maven and ivy packages, and it seems like only Artifactory supports both. I went through the installation and got things roughly working, except that I couldn't point my remote repository (for local caching) at e.g., sbt-plugin-releases because it uses a custom repository layout (including the sbt version), which isn't allowed in the free version of Artifactory.

StackOverflow

return in a scala function literal

I'm trying to do something like this:

var fun : (Int,Int) => Double = (a,b) =>
{
// do something
return 1.0
}


However, my IDE complaints with Return statement outside method definition. So how do I explicitly give a return statement in a function literal in scala?

How does Spray's "jsonFormatX" work?

I can convert my case class to json using the Spray-Json library by doing this:

case class Foo(a1:Int, a2:Int)
...
implicit val fooFormat = jsonFormat2(Foo) // notice the "2" at the end
...


Looking through the source code, I don't see anything like def jsonFormat1... def jsonFormat2... jsonFormat3.. and so on... How are the developers of this library doing this type of thing? See here

Planet Theory

Improved ancestry labeling scheme for trees

Authors: Søren Dahlgaard, Mathias Bæk Tejs Knudsen, Noy Rotbart