# Planet Primates

## December 06, 2013

### StackOverflow

#### How to force Intellij IDEA Scala/SBT plugin to reread dependencies from build.sbt?

I'm using Intellij IDEA 13 (community edition) with the Scala plugin.

My initial import of an existing Scala project with a build.sbt worked fine. The library dependencies were picked up by IDEA. Additional dependencies added after the initial import were not picked up, although I had checked the "auto-import" option.

How can I force IDEA to reload the dependencies from build.sbt?

#### Is there a way to make a function var dynamic after definition?

I'm using clojure.test in the context of "Clojure in Action/Chapter 8 - Test driven development" and wrote some code for stubbing and mocking function implementations. The stubbing/mocking is done through macros generating a "binding" form.

The code was written assuming some previous version of Clojure where every var could be bound with a "binding" form. So, when I try to run a calling to the macro I get "Can't dynamically bind non-dynamic var".

Defining the functions as dynamic solves the problem.

Since the stubbing/mocking of a function seems very useful but most functions are not dynamic, it would make sense to make them dynamic, at least just for testing.

I tried:

(def ^{:dynamic true} log-call #'log-call)


where log-call is the symbol naming the var holding the function.

And it worked just as expected. The binding produced by the macro treated my function log-call as dynamic.

When trying the same approach with a different function, I started getting the error:

StackOverflowError   clojure.lang.Var.fn (Var.java:392)


I'm not really sure why it's happening and don't find a pattern neither.

So, the questions?

1) Is my approach of "(def ^{:dynamic true} log-call #'log-call)" valid for making a function var dynamic after definition?

2) Is there another way available?

UPDATED:

3) What is wrong with my code bellow? I managed to abstract the problem from my code.

If you remove line

(def ^{:dynamic true} log-call #'log-call)


it works just fine. So, why adding the line produces

StackOverflowError   clojure.lang.Var.deref (Var.java:207)


Here is the code:

(def all-expenses [{:amount 33.0 :date "today"}
{:amount 44.0 :date "yesterday"}])

(defn log-call [id]
(println "Audit - called" id))

(defn fetch-all-expenses []
(log-call "fetch-all"))

(defn expenses-greater-than [expenses threshold]
(log-call "expenses-greater-than")
expenses)

(defn fetch-expenses-greater-than [threshold]
(let [all (fetch-all-expenses)]
(expenses-greater-than all threshold)))

(def ^{:dynamic true} log-call #'log-call)

(def ^{:dynamic true} fetch-all-expenses #'fetch-all-expenses)

(binding [fetch-all-expenses (constantly all-expenses)]
(let [filtered (fetch-expenses-greater-than 15.0)]
(println (= (count filtered) 2))))


#### Polymorphism with Scala type classes

We are refactoring an inherited method to use a type class instead, but we're running into some trouble as we're fairly new to type classes. At present method is defined as

trait MethodTrait {
def method: Map[String, Any] = // default implementation
}

abstract class SuperClass extends MethodTrait {
override def method: super.method ++ // SuperClass implementation
}

class Clazz extends SuperClass {
override def method: super.method ++ // Clazz implementation
}


and so on, where there are a total of 50+ concrete classes, the hierarchy is fairly shallow (abstract class SuperClass -> abstract class SubSuperClass -> abstract class SubSubSuperClass -> class ConcreteClass is as deep as it goes), and a concrete class never extends another concrete class. (In the actual implementation, method returns a Play Framework JsObject instead of a Map[String, Any].) We're trying to replace this with a type class:

trait MethodTrait[T] {
def method(target: T): Map[String, Any]
}

class MethodType {
type M[T] = MethodTrait[T]
}

implicit object Clazz1Method extends MethodTrait[Clazz1] {
def method(target: Clazz1): Map[String, Any] { ... }
}

implicit object Clazz2Method extends MethodTrait[Clazz2] {
def method(target: Clazz2): Map[String, Any] { ... }
}

// and so on


I'm running into two problems:

A. Mimicking the super.method ++ functionality from the previous implementation. At present I'm using

class Clazz1 extends SuperClass

class Clazz2 extends SubSuperClass

private def superClassMethod(s: SuperClass): Map[String, Any] { ... }

private def subSuperClassMethod(s: SubSuperClass): Map[String, Any] {
superClassMethod(s) ++ ...
}

implicit object Clazz1Method extends MethodTrait[Clazz1] {
def method(target: Clazz1): Map[String, Any] {
superClassMethod(target) ++ ...
}
}

implicit object Clazz2Method extends MethodTrait[Clazz2] {
def method(target: Clazz2): Map[String, Any] {
subSuperClassMethod(target) ++  ...
}
}


but this is ugly, and I won't get a warning or error if I accidentally call a method too far up the hierarchy e.g. if Clazz2 calls superClassMethod instead of subSuperClassMethod.

B. Calling method on a superclass, e.g.

val s: SuperClass = new Clazz1()
s.method


Ideally I'd like to be able to tell the compiler that every subclass of SuperClass has a corresponding implicit object for method in the type class and so s.method is type-safe (or I'll get a compile time error if I've neglected to implement a corresponding implicit object for a subclass of SuperClass), but instead all I've been able to come up with is

implicit object SuperClassMethod extends MethodTrait[SuperClass] {
def method(target: SuperClass): Map[String, Any] {
target match {
case c: Clazz1 => c.method
case c: Clazz2 => c.method
...
}
}
}


which is ugly and won't give me a compile-time warning or error if I've omitted a class since I can't define SuperClass as a sealed trait.

### StackOverflow

#### Initial value in loops

Suppose you are writing a tail-recursive loop function to evaluate a collection of elements according to some criterium and want to end up with the element that scores best, and its score.

Naturally you will pass the best scoring element so far, as well as its score, as parameters to the function.

But since there is no best element at the start of the recursion, what should you initially pass as parameters to the loop function? Not wanting to use null, you could use Option[T] as parameter types, but then you have to check for isEmpty at each recursion while you know that it always has a value after the initial call. Isn't there a better way?

### StackOverflow

#### object Database is not a member exception on slick typesafe extensions for oracle.

I am trying to port scala/angular tutorial (https://github.com/pcleary00/play-angularjs) over to oracle using typesafe extensions for slick. Really appreciate any help/guidance/x I did the following changes: 1. added library dependencies to build.scala

resolvers += "Typesafe Releases" at "http://repo.typesafe.com/typesafe/maven-releases/"
val appDependencies = Seq(
jdbc,
"com.typesafe.slick" %% "slick" % "2.0.0-M3",
"com.typesafe.play" %% "play-slick" % "0.5.0.8",
"com.typesafe.slick" %% "slick-extensions" % "2.0.0-M3",
"org.joda" % "joda-money" % "0.9"
)


Then application.conf

db.default.driver=oracle.jdbc.driver.OracleDriver
db.default.url="jdbc:oracle:thin:host-name:port/sid"
db.default.user=scott


Then imports:

import com.typesafe.slick.driver.oracle.OracleDriver.simple._
import scala.slick.driver.ExtendedDriver
import scala.slick.lifted.ColumnOption.PrimaryKey
import scala.slick.session.Database
import play.api.db.DB
import play.api.Play.current


Looks like it was able to find the extensions but I got the following errors:

[info] Compiling 8 Scala sources and 1 Java source to C:\tmp\play-angularjs-master\target\scala-2.10\classes...
[error] C:\tmp\play-angularjs-master\app\Global.scala:4: object Database is not a member of package scala.slick.session
[error] import scala.slick.session.Database
[error]        ^
[error] C:\tmp\play-angularjs-master\app\Global.scala:7: object Database is not a member of package scala.slick.session


#### How would a database be created using clojure's java.jdbc?

I want to be able to write the following functions for jdbc:

(def db {:classname "com.mysql.jdbc.Driver"
:subprotocol "mysql"
:subname "//127.0.0.1:3306/santacsv"
:server "//127.0.0.1:3306"
:schema "santa"
:user "root"

(defn has-schema? [db & [schema]])

(defn create-schema [db & [schema]])

(defn drop-schema [db & [schema]])


I'm not sure how to go about it. Any help would be appreciated

#### is it possible to overcome type erasure in akka receive?

I have a trait and implementing akka actor that can't properly differentiate the types of incoming messages due to type erasure, and so the first case is matching all messages.

I'm using scala 2.10.x, and from reading many other answers I know that it is possible to recover type information inside a trait using TypeTag or ClassTag, but I can't figure out how to apply it (if it is possible) within the akka receive.

My (very simplified) example is as follows. Is it possible properly match the generic types?

import akka.actor.Actor
import akka.actor.ActorSelection

trait MessageProcessor[A,B] {
this: Actor =>

val destA: ActorSelection
val destB: ActorSelection

def process(a: A): A
def process(a: B): B

case a: A =>
destA ! process(a)
case b: B =>
destB ! process(b)
}
}

class StringIntProcessor(val destA: ActorSelection, val destB: ActorSelection) extends MessageProcessor[String,Int] with Actor {
def process(a: String) = { a + "1" }
def process(b: Int) = { b + 1 }
}


### TheoryOverflow

#### Parameterized complexity from P to NP-hard and back again

I'm looking for examples of problems parametrized by a number $k \in \mathbb{N}$, where the problem's hardness is non-monotonic in $k$. Most problems (in my experience) have a single phase transition, for example $k$-SAT has a single phase transition from $k \in \{1,2\}$ (where the problem is in P) to $k \ge 3$ (where the problem is NP-complete). I'm interested in problems where there are phase transitions in both directions (from easy to hard and vice-versa) as $k$ increases.

My question is somewhat similar to the one asked at Hardness Jumps in Computational Complexity, and in fact some of the responses there are relevant to my question.

Examples I am aware of:

1. $k$-colorability of planar graphs: In P except when $k=3$, where it is NP-complete.
2. Steiner tree with $k$ terminals: In P when $k=2$ (collapses to shortest $s$-$t$ path) and when $k=n$ (collapses to MST), but NP-hard "in between". I don't know if these phase transitions are sharp (e.g., P for $k_0$ but NP-hard for $k_0+1$). Also the transitions of $k$ depend on the size of input instance, unlike my other examples.
3. Counting satisfying assignments of a planar formula modulo $n$: In P when $n$ is a Mersenne prime number $n=2^k-1$, and #P-complete for most(?)/all other values of $n$ (from Aaron Sterling in this thread). Lots of phase transitions!
4. Induced subgraph detection: The problem is not parametrized by an integer but a graph. There exist graphs $H_1 \subseteq H_2 \subseteq H_3$ (where $\subseteq$ denotes a certain kind of subgraph relation), for which determining whether $H_i \subseteq G$ for a given graph $G$ is in P for $i\in \{1,3\}$ but NP-complete for $i=2$. (from Hsien-Chih Chang in the same thread).

### CompsciOverflow

#### how to connect two pc's behind routers

Setup: I have two routers , one is next to inline cable modem. There is one pc (192.168.2.102) off this router. The router then has a cable to other side of the house that goes to another router. That router has a pc behind it (192.168.6.2). The 2.102 can ping and connect to servers on the .6.2 but it wont work the other way around. How can I resolve this so it works both ways?

### Lambda the Ultimate Forum

#### Limitations of Prolog-style clausal programs

Beginning in the 1970’s, Japan took the DRAM market (and consequently most of the integrated circuit industry) away from the previous US dominance. This was accomplished with the help of the Japanese VLSI project that was funded and coordinated in good part by the Japanese government Ministry of International Trade and Industry (MITI) [Sigurdson 1986].

MITI hoped to repeat this victory by taking over the computer industry. However, Japan had come under criticism for “copying” the US. One of the MITI goals for ICOT was to show that Japan could innovate new computer technology and not just copy the Americans.

ICOT tried to go all the way with Prolog-style clause programs. Kowalski later recalled “Having advocated LP [Logic Programming] as a unifying foundation for computing, I was delighted with the LP [Logic Programming] focus of the FGCS [Fifth Generation Computer Systems] project.” [Fuchi, Kowalski, Ueda, Kahn, Chikayama, and Tick 1993] By making Prolog-style clause programming (which was mainly being developed outside the US) the foundation, MITI hoped that the Japanese computer industry could leapfrog the US. “The [ICOT] project aimed to leapfrog over IBM, and to a new era of advanced knowledge processing applications” [Sergot 2004]

Unfortunately, ICOT misjudged the importance of the Actor Model [Hewitt, Bishop, and Steiger 1973] which had been developed in reaction to the limitations of Planner. ICOT had to deal with the concurrency and consequently developed concurrent programming languages based on Prolog-style clauses [Shapiro 1989].

However, it proved difficult to implement clause invocation in these languages as efficiently as message-passing in object-oriented programming languages. Simula-67 originated a hierarchical class structure for objects so that message handling procedures (methods) and object instance variables could be inherited by subclasses. Ole-Johan Dahl [1967] invented a powerful compiler technology using dispatch tables that enabled message handling procedures in subclasses of objects to be efficiently invoked.
The combination of efficient inheritance-based procedure invocation together with class libraries and browsers (pioneered in Smalltalk) provided better tools than the slower pattern-directed clause invocation of the FGCS Prolog-style clause programming languages. Consequently, the ICOT programming languages never took off and instead object-oriented like Java and C# became the mainstream.

Going beyond object-oriented programming to concurrent programming, ICOT encountered further difficulties dealing with concurrency, e.g., readers-writers concurrency. Concurrency control for readers and writers in a shared resource is a classic problem. The fundamental constraint is that multiple writers are not allowed to operate concurrently. Below is an implementation of a reading priority guardian that encapsulates a shared resource:

CreateReadingPriority.[theResource:ReadersWriter] ≡
Actor with writing≔False, numberReading:(Integer thatIs ≧0)≔0◊
Enqueue (writing or not IsEmpty writersQ) ¿ True ↝ readersQ; False ↝ NullQueue ?
RequirePre not writing◊
always permit (IsEmpty writersQ) ¿
False ↝ (numberReading = 1) ¿
True ↝ writersQ;
False ↝ NullQueue ? ?
Write[update]→
Enqueue (numberReading>0 or not IsEmpty readersQ or writing or not IsEmpty writersQ) ¿
True ↝  writersQ;
False ↝ NullQueue ?
RequirePre not writing◊  Prep writing≔True◊
Hole theResource.Write[update]
True ↝ writersQ;
also RequirePre writing◊ writing≔False◊§▮


Implementations of the above kind procedure in the Prolog-style clausal languages used by ICOT were both verbose and inefficient.

The technical managers at ICOT were aware of some of the pitfalls that had tripped up previous Artificial Intelligence (AI) researchers. So they deliberately avoided calling ICOT an AI Project. Instead they had the vision of an integrated hardware/software system [Uchida and Fuchi 1992]. However, the Prolog-style clause programming paradigm turned not to be a suitable foundation because of poor modularity and lack of efficiency by comparison with direct message passing [Hewitt and Agha 1988].

Another problem was that multi-processors found it difficult to compete because at the time single processors were rapidly increasing in speed and connections between multiple processors suffered long latencies.

The MITI strategy backfired because Japanese companies refused to productize ICOT hardware.

However, the architects of ICOT did get some things right:
* The project largely avoided the Mental Agent paradigm
* The project correctly placed tremendous emphasis on research in concurrency and parallelism as an emerging computing paradigm.

The architectural reliance on Prolog-style clausal programs was a principle contributing cause for the failure of ICOT to achieve commercial success.

### StackOverflow

#### Understanding scala slick

Now, I'm writing sample for learning scala slick. I'm using some github projs and stackoverflow (Q&A)s. Below my sample code:

import scala.slick.driver.PostgresDriver.simple._

object TestApp extends App{

case class MyTable(id: Option[Int], foo: String, bar: String)

object MyTables extends Table[MyTable]("mytable") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def foo = column[String]("foo", O.NotNull)
def bar = column[String]("bar", O.NotNull)
def * = id.?  ~ foo ~ bar <> (MyTable, MyTable.unapply _)
def forInsert = foo ~ bar <>
({ (f, l) => MyTable (None, f, l) }, { ep:MyTable => Some((ep.foo, ep.bar)) })
val findByID = createFinderBy(_.id)
}

implicit val session = Database.forURL("jdbc:postgresql://localhost:5432/myserver",
driver="org.postgresql.Driver",
session withTransaction {
MyTables.ddl.create
MyTables.foo ~ MyTables.bar).insert("Homer", "Simpson")
MyTables.forInsert.insertAll(
MyTable(None, "Marge", "Bouvier"),
MyTable(None, "Carl", "Carlson"),
MyTable(None, "Lenny", "Leonard")
)
}

}


EXCEPTION:

Exception in thread "main" java.lang.NoClassDefFoundError: scala/Right at scala.slick.driver.BasicProfile$class.createQueryTemplate(BasicProfile.scala:12) at scala.slick.driver.PostgresDriver$.createQueryTemplate(PostgresDriver.scala:69) at scala.slick.ql.Parameters.flatMap(Parameters.scala:9) at scala.slick.driver.BasicTableComponent$Table.createFinderBy(BasicTableComponent.scala:30) at TestApp$MyTables$.(TestApp.scala:16) Create table in postgresql CREATE TABLE mytable ( id serial primary key, foo VARCHAR(40) not null, bar VARCHAR(40) not null, );  I'm using this tools and libraries: 1. Scala IDE - 2.10 2. Java version - 1.7.0_11 3. slick_2.10.0-M4-0.10.0-M2.jar 4. postgresql-9.2-1003-jdbc4.jar 5. Database - PostgreSQL 8.3 What's Wrong Here? Thanks in advance. #### how to do sequential execution of Futures in scala I have this scenario where I need to use an iterator, for each of the item a function f(item) is called and returns a Future.Unit. however, I need to make it that each f(item) call is executed sequentially. they can not run in parallel. for(item <- it) f(item)  won't work becuase this starts all the calls in parallel. How do I do it so they follow in sequence? #### Combining Classes and Event Listeners: this keyword I'm having a hard time answering my question, and I think it's simply because "class" and "this" and other such terms are too generic for effective Googling. Consider the following code: function product (name) { var _productName; this.getProductName = function() { return this._productName; }; this.alertProductName = function() { // This errors because this is a reference to the HTML <input> tag and not the class alert(this.getProductName()); } this._productName = name; var$productButton = $('<input type="button" value="What is my name?" />');$productButton.on('click', this.alertProductName);

$('body').append($productButton);
}

var widget = new product('Widget');



jQuery (or maybe Javascript itself) is resetting what the this keyword points to once product::alertProductName is called as a callback to an event. I can't find any other way to access a specific instance of this class from within a function used as a callback. It looks like Javascript used to have arguments.callee but that has been deprecated.

Does anyone know how I can access a specific instance of a class in this manner? If not, is there a better way to write this so that I don't have this problem to begin with?

### CompsciOverflow

#### Non-uniform random distribution: How do I get a random between 100 and 180 that is on average close to 120? (like in a Gaussian distribution)

Presume I have the following case:

• int value a
• int value b, for which a < b
• int value i, for which a <= i < b
• I need an int value x, for which a <= x < b, randomly chosen, according to a non-uniform distribution: the distribution should follow a bell curve which is centered around i.

In Java, I have the following methods available on java.util.Random:

• nextInt(int m): int between 0 and m
• nextDouble(): double between 0.0 and 1.0
• nextGaussian(): double between -Infinity and +Infinity, usually close to 0.0.

How do I build such a non-uniform distribution from these building blocks? How do I reliably and efficiently transform nextGaussian() into nextGaussian(a, b, i)? The part I am struggling with is to enforce that x is selected between a and b (without doing a trial-and-error).

### StackOverflow

#### Scala: return has its place

EDIT3
This is the "final" solution, again thanks to Dan Burton.

def save = Action { implicit request =>
val result = for {
model   <- bindForm(form).right // error condition already json'd
transID <- payment.process(model, orderNum) project json
userID  <- dao.create(model, ip, orderNum, transID) project json
} yield (userID, transID)
}


Then the pimp'd Either project method, placed somewhere in your application (in my case, an implicits trait that sbt root & child project(s) extends their base package object from:

class EitherProvidesProjection[L1, R](e: Either[L1, R]) {
def project[L1, L2](f: L1 => L2) = e match {
case Left(l:L1) => Left(f(l)).right
case Right(r)   => Right(r).right
}
}
@inline implicit final def either2Projection[L,R](e: Either[L,R]) = new EitherProvidesProjection(e)


EDIT2
Evolution, have gone from embedded return statements to this little white dwarf of density (kudos to @DanBurton, the Haskell rascal ;-))

def save = Action { implicit request =>
val result = for {
model   <- form.bindFromRequest fold(Left(_), Right(_)) project( (f:Form) => Conflict(f.errorsAsJson) )
transID <- payment.process(model, orderNum) project(Conflict(_:String))
userID  <- dao.create(model, ip, orderNum, transID) project(Conflict(_:String))
} yield (userID, transID)
...
}


I have added Dan's onLeft Either projection as a pimp to Either, with the above "project" method, which allows for right-biased eitherResult project(left-outcome). Basically you get fail-first error as a Left and success as a Right, something that would not work when feeding Option outcomes to for comprehension (you only get Some/None outcome).

The only thing I'm not thrilled with is having to specify the type for the project(Conflict(param)); I thought the compiler would be able to infer the left condition type from the Either that is being passed to it: apparently not.

At any rate, it's clear that the functional approach obviates the need for embedded return statements as I was trying to do with if/else imperative approach.

EDIT
The functional equivalent is:

val bound = form.bindFromRequest
bound fold(
error=> withForm(error),
model=> {
val orderNum = generateOrderNum()
payment.process(model, orderNum) fold (
whyfail=> withForm( bound.withGlobalError(whyfail) ),
transID=> {
dao.createMember(model, ip, orderNum, transID) fold (
errcode=>
Ok(withForm( bound.withGlobalError(i18n(errcode)) )),
userID=>
// generate pdf, email, redirect with flash success
)}
)}
)


which is certainly a densely power packed block of code, a lot happening there; however, I would argue that corresponding imperative code with embedded returns is not only similarly concise, but also easier to grok (with added benefit of fewer trailing curlies & parens to keep track of)

ORIGINAL
Finding myself in an imperative situation; would like to see an alternative approach to the following (which does not work due to the use of return keyword and lack of explicit type on method):

def save = Action { implicit request =>
val bound = form.bindFromRequest
if(bound.hasErrors) return Ok(withForm(bound))

val model = bound.get
val orderNum = generateOrderNum()
val transID  = processPayment(model, orderNum)
if(transID.isEmpty) return Ok(withForm( bound.withGlobalError(...) ))

val result = dao.createMember(model, ip, orderNum, transID)
result match {
case Left(_) =>
Ok(withForm( bound.withGlobalError(...) ))
case Right((foo, bar, baz)) =>
// all good: generate pdf, email, redirect with success msg
}
}
}


In this case I like the use of return as you avoid nesting several if/else blocks, or folds, or matches, or fill-in-the-blank non-imperative approach. The problem of course, is that it doesn't work, an explicit return type has to specified, which has its own issues as I have yet to figure out how to specify a type that satisfies whatever Play magic is at work -- no, def save: Result, does not work as the compiler then complains about implicit result now not having an explicit type ;-(

At any rate, Play framework examples provide la, la, la, la happy 1-shot-deal fold(error, success) condition which is not always the case in the real world™ ;-)

So what is the idiomatic equivalent (without use of return) to above code block? I assume it would be nested if/else, match, or fold, which gets a bit ugly, indenting with each nested condition.

#### How do I parse a x-www-url-encoded string into a Map[String, String] using Lift?

From Lift, I'm getting a string of the form

TOKEN=EC%2d454178600K772032D&TIMESTAMP=2011%2d06%2d29T13%3a10%3a58Z&CORRELATIONID=cbd56e97cad38&ACK=Success&VERSION=64&BUILD=1936884


from the response of an HTTP request.

Although it's probably ultra-trivial, I can't find the Lift function that parses this into a nice Map[String, String]. Any help?

### infra-talk

#### Install Homebrew on OS X Mavericks

Install homebrew from http://brew.sh on your shiny new OS X machine.

### QuantOverflow

#### Single Most Important Fact about a Fund - Interview Question

I recently attended an interview to work as a software developer in an Asset Management company. I was asked by the interviewer:

What is the most important piece of information that should be reviewed before investing in a fund?

I highlighted that a low expense ratio, low turnover and low fees are the most important things to look. But this is not what the interviewer was looking for in my answer. I prompted him further to find out but he wouldn't say.

Is there anything else more important that what i have mentioned when investing in a fund? I cant think of anything else apart from current performance against a benchmark?

### Planet Theory

#### Quasi-Polynomial Time Approximation Scheme for Sparse Subsets of Polygons

Authors: Sariel Har-Peled
Abstract: We describe how to approximate, in quasi-polynomial time, the largest independent set of polygons, in a given set of polygons. Our algorithm works by extending the result of Adamaszek and Wiese \cite{aw-asmwi-13, aw-qmwis-14} to polygons of arbitrary complexity. Surprisingly, the algorithm also works or computing the largest subset of the given set of polygons that has some sparsity condition. For example, we show that one can approximate the largest subset of polygons, such that the intersection graph of the subset does not contain a cycle of length $4$ (i.e., $K_{2,2}$).

#### Persistent homotopy types of noisy images

Authors: Vitaliy Kurlin
Abstract: We present a fast algorithm to identify the topological shape (homotopy type) of a noisy dotted image in the plane. The algorithm has $O(n\log n)$ time and $O(n)$ space in the number $n$ of points in a given image. The only input is a point cloud. The output is the number of all non-trivial loops that persist (have a long life span) when the image is analyzed at all possible scales. We give theoretical guarantees when the algorithm correctly identifies the homotopy type by using only a noisy sample of a triangulable set.

#### Outerstring graphs are $\chi$-bounded

Authors: Alexandre Rok, Bartosz Walczak
Abstract: An outerstring graph is an intersection graph of curves lying in a halfplane with one endpoint on the boundary of the halfplane. It is proved that the outerstring graphs are $\chi$-bounded, that is, their chromatic number is bounded by a function of their clique number. This generalizes a series of previous results on $\chi$-boundedness of outerstring graphs with various restrictions of the shape of the curves or the number of times the pairs of curves can intersect. This also implies that the intersection graphs of $x$-monotone curves with bounded clique number have chromatic number $O(\log n)$, improving the previous polylogarithmic upper bound. The assumption that each curve has an endpoint on the boundary of the halfplane is justified by the known fact that triangle-free intersection graphs of straight-line segments can have arbitrarily large chromatic number.

#### Approximating persistent homology for a cloud of $n$ points in a subquadratic time

Authors: Vitaliy Kurlin
Abstract: The Vietoris-Rips filtration for an $n$-point metric space is a sequence of large simplicial complexes adding a topological structure to the otherwise disconnected space. The persistent homology is a key tool in topological data analysis and studies topological features of data that persist over many scales. The fastest algorithm for computing persistent homology of a filtration has time $O(M(u)+u^2\log^2 u)$, where $u$ is the number of updates (additions or deletions of simplices), $M(u)=O(u^{2.376})$ is the time for multiplication of $u\times u$ matrices. For a space of $n$ points given by their pairwise distances, we approximate the Vietoris-Rips filtration by a zigzag filtration consisting of $u=o(n)$ updates, which is sublinear in $n$. The constant depends on a given error of approximation and on the doubling dimension of the metric space. Then the persistent homology of this sublinear-size filtration can be computed in time $o(n^2)$, which is subquadratic in $n$.

#### Fast Subspace Approximation via Greedy Least-Squares

Authors: Mark Iwen, Felix Krahmer
Abstract: In this note, we develop fast and deterministic dimensionality reduction techniques for a family of subspace approximation problems. Let $P\subset \mathbbm{R}^N$ be a given set of $M$ points. The techniques developed herein find an $O(n \log M)$-dimensional subspace that is guaranteed to always contain a near-best fit $n$-dimensional hyperplane $\mathcal{H}$ for $P$ with respect to the cumulative projection error $(\sum_{{\bf x} \in P} \| {\bf x} - \Pi_\mathcal{H} {\bf x} \|^p_2)^{1/p}$, for any chosen $p > 2$. The deterministic algorithm runs in $\tilde{O} (MN^2)$-time, and can be randomized to run in only $\tilde{O} (MNn)$-time while maintaining its error guarantees with high probability. In the case $p = \infty$ the dimensionality reduction techniques can be combined with efficient algorithms for computing the John ellipsoid of a data set in order to produce an $n$-dimensional subspace whose maximum $\ell_2$-distance to any point in the convex hull of $P$ is minimized. The resulting algorithm remains $\tilde{O} (MNn)$-time. In addition, the dimensionality reduction techniques developed herein can also be combined with other existing subspace approximation algorithms for $2 < p \leq \infty$ - including more accurate algorithms based on convex programming relaxations - in order to reduce their runtimes.

#### On the number of edges in families of pseudo-discs

Authors: Piotr Micek, Rom Pinchasi
Abstract: We give a simple argument showing that the number of edges in the intersection graph G of a family of n pseudo-discs is at most $(3e+1)\omega(G)n$, where $\omega(G)$ is the clique number of G. In particular, it follows that $\chi(G)\leq col(G) < 19\omega(G)$. This generalizes previous results for intersection graphs of translates and homothethic copies of a fixed convex set in the plane.

#### Orienting Fully Dynamic Graphs with Worst-Case Time Bounds

Authors: Tsvi Kopelowitz, Robert Krauthgamer, Ely Porat, Shay Solomon
Abstract: In edge orientations, the goal is usually to orient (direct) the edges of an undirected $n$-vertex graph $G$ such that all out-degrees are bounded. When the graph $G$ is fully dynamic, i.e., admits edge insertions and deletions, we wish to maintain such an orientation while keeping a tab on the update time. Low out-degree orientations turned out to be a surprisingly useful tool, with several algorithmic applications involving static or dynamic graphs.

Brodal and Fagerberg (1999) initiated the study of the edge orientation problem in terms of the graph's arboricity, which is very natural in this context. They provided a solution with constant out-degree and \emph{amortized} logarithmic update time for all graphs with constant arboricity, which include all planar and excluded-minor graphs. However, it remained an open question (first proposed by Brodal and Fagerberg, later by others) to obtain similar bounds with worst-case update time.

We resolve this 15 year old question in the affirmative, by providing a simple algorithm with worst-case bounds that nearly match the previous amortized bounds. Our algorithm is based on a new approach of a combinatorial invariant, and achieves a logarithmic out-degree with logarithmic worst-case update times. This result has applications in various dynamic graph problems such as maintaining a maximal matching, where we obtain $O(\log n)$ worst-case update time compared to the $O(\frac{\log n}{\log\log n})$ amortized update time of Neiman and Solomon (2013).

#### Efficient construction of the lattice of frequent closed patterns and simultaneous extraction of generic bases of rules

Authors: Tarek Hamrouni, Sadok Ben Yahia, Engelbert Mephu Nguifo
Abstract: In the last few years, the amount of collected data, in various computer science applications, has grown considerably. These large volumes of data need to be analyzed in order to extract useful hidden knowledge. This work focuses on association rule extraction. This technique is one of the most popular in data mining. Nevertheless, the number of extracted association rules is often very high, and many of them are redundant. In this paper, we propose a new algorithm, called PRINCE. Its main feature is the construction of a partially ordered structure for extracting subsets of association rules, called generic bases. Without loss of information these subsets form representation of the whole association rule set. To reduce the cost of such a construction, the partially ordered structure is built thanks to the minimal generators associated to frequent closed patterns. The closed ones are simultaneously derived with generic bases thanks to a simple bottom-up traversal of the obtained structure. The experimentations we carried out in benchmark and "worst case" contexts showed the efficiency of the proposed algorithm, compared to algorithms like CLOSE, A-CLOSE and TITANIC.

#### Quantum Communications Based on Quantum Hashing

Authors: Alexander Vasiliev
Abstract: Based on the polynomial presentation of Boolean functions and quantum hashing technique we present a method for computing Boolean functions in the quantum one-way communication model. Some of the results are also true in a more restricted Simultaneous Message Passing model.

#### On the relation generation method of Joux for computing discrete logarithms

Authors: Ming-Deh Huang, Anand Kumar Narayanan
Abstract: In \cite{joux}, Joux devised an algorithm to compute discrete logarithms between elements in a certain subset of the multiplicative group of an extension of the finite field $\mathbb{F}_{p^n}$ in time polynomial in $p$ and $n$. Shortly after, Barbulescu, Gaudry, Joux and Thome \cite{bgjt} proposed a descent algorithm that in $(p n)^{\mathcal{O}(\log n)}$ time projects an arbitrary element in $\mathbb{F}_{p^n}^\times$ as a product of powers of elements in the aforementioned subset. Together, these two algorithms yield a quasi-polynomial time algorithm for computing discrete logarithms in finite fields of small characteristic. The success of both the algorithms are reliant on heuristic assumptions. We identify obstructions that prevent certain heuristic assumptions they make from being true in general. Further, we describe methods to overcome these obstructions.

### StackOverflow

#### How should I communicate with a server?

Imagine some fellow wants to query a pizza server for the list of pizzas. This individual would do simply

 GET /pizzas
;=> ["cheese", "extra cheese", "broccoli"]


With pedestal-app's data model and messages, I am not sure how to design client-server communication. Here are the possibilities some minutes of hammocking brought:

1. An effect-consumer that
• transforms a message into an HTTP request
• transforms back the results (to e.g. [{:type :add :topic [:pizzas] :value "cheese"} ...])
• puts the messages in the queue
2. A dedicated resource on the server (e.g. "/edn") that
• accepts pedestal messages
• dispatches to the right function
• responds with the raw data (i.e. ["cheese", "extra cheese", "broccoli"])
• has the effect-consumer transform back the results to messages
3. A dedicated resource that uses the routes. Just like #2, but
• altering the request
• forwarding it to another entry in route table
4. Messages on both sides, with
• the server transforming messages into function calls
• the server transforming results back into messages
• the client just adding these messages to the queue

It seems to me that with approaches #2 and #4, I'd bypass and lose all the benefit of the interceptors. With approach #2, I'd need to redouble the routing logic. With approach #4, I'd also need to generate a lot of code to accommodate the pedestal client.

Options #1 and #3 seem better, but #3 smells hacky and #1, misdirected.

How are you guys doing it?

Thanks!

### arXiv Networking and Internet Architecture

#### Simplicial homology for future cellular networks. (arXiv:1312.1664v1 [cs.NI])

In this article, we apply the simplicial complex representation to cellular networks, then using a reduction algorithm for simplicial complexes we provide algorithms for the management of future cellular networks. First, we propose a frequency auto-planning algorithm for the self-configuration of future cellular networks. It aims at minimizing the number of planned frequencies while maximizing the usage of each one. Then, we present an energy conservation algorithm for the self- optimization of future cellular networks during off-peak hours. It takes into account not only coverage, but also user traffic. Finally, we present and discuss the performance of a disaster recovery algorithm using determinantal point processes to patch coverage holes.

#### Detection and prevention of botnets and malware in an enterprise network. (arXiv:1312.1629v1 [cs.CR])

One of the most significant threats faced by enterprise networks today is from Bots. A Bot is a program that operates as an agent for a user and runs automated tasks over the internet, at a much higher rate than would be possible for a human alone. A collection of Bots in a network, used for malicious purposes is referred to as a Botnet. Bot attacks can range from localized attacks like key-logging to network intensive attacks like Distributed Denial of Service (DDoS). In this paper, we suggest a novel approach that can detect and combat Bots. The proposed solution adopts a two pronged strategy which we have classified into the standalone algorithm and the network algorithm. The standalone algorithm runs independently on each node of the network. It monitors the active processes on the node and tries to identify Bot processes using parameters such as response time and output to input traffic ratio. If a suspicious process has been identified the network algorithm is triggered. The network algorithm will then analyze conversations to and from the hosts of the network using the transport layer flow records. It then tries to deduce the Bot pattern as well as Bot signatures which can subsequently be used by the standalone algorithm to thwart Bot processes at their very onset.

#### Instruction sequence expressions for the Karatsuba multiplication algorithm. (arXiv:1312.1529v1 [cs.PL])

The Karatsuba multiplication algorithm is an algorithm for computing the product of two natural numbers represented in the binary number system. This means that the algorithm actually computes a function on bit strings. The restriction of this function to bit strings of any given length can be computed according to the Karatsuba multiplication algorithm by a finite instruction sequence that contains only instructions to set and get the content of Boolean registers, forward jump instructions, and a termination instruction. We describe the instruction sequences concerned for the restrictions to bit strings of the different lengths by uniform terms from an algebraic theory.

#### Efficient construction of broadcast graphs. (arXiv:1312.1523v1 [cs.DM])

A broadcast graph is a connected graph, $G=(V,E)$, $|V |=n$, in which each vertex can complete broadcasting of one message within at most $t=\lceil \log n\rceil$ time units. A minimum broadcast graph on $n$ vertices is a broadcast graph with the minimum number of edges over all broadcast graphs on $n$ vertices. The cardinality of the edge set of such a graph is denoted by $B(n)$. In this paper we construct a new broadcast graph with

$B(n) \le (k+1)N -(t-\frac{k}{2}+2)2^{k}+t-k+2$, for $n=N=(2^{k}-1)2^{t+1-k}$ and

$B(n) \le (k+1-p)n -(t-\frac{k}{2}+p+2)2^{k}+t-k -(p-2)2^{p}$, for $2^{t} < n<(2^{k}-1)2^{t+1-k}$, where $t \geq 7$, $2 \le k \le \lfloor t/2 \rfloor -1$ for even $n$ and $2 \le k \le \lceil t/2 \rceil -1$ for odd $n$, $d=N-n$, $x= \lfloor \frac{d}{2^{t+1-k}} \rfloor$ and $p = \lfloor \log_{2}{(x+1)} \rfloor$ if $x>0$ and $p=0$ if $x=0$.

The new bound is an improvement upon the bound presented by Harutyunyan and Liestman (2012) for odd values of $n$.

#### Towards the Framework of Information Security. (arXiv:1312.1460v1 [cs.CR])

Todays modern society is extremely dependent on computer based information systems. Many of the organizations would simply not be able to function properly without services provided by these systems, just like financing organizations. Although interruption might decrease the efficiency of an organization, theft or unintentional disclosure of entrusted private data could have more serious consequences, such as legal actions as well as loss of business due to lack of trust from potential users. This dependence on information systems has lead to a need for securing these systems and this in turn has created a need for knowing how secure they are. The introduction of the information society has changed how people interact with government agencies. Government agencies are now encouraged to uphold a 24-hour electronic service to the citizens. The introduction of government services on the Internet is meant to facilitate communication with agencies, decrease service times and to lessen the amount of papers that needs to be processed. The increased connectivity to the Internet results in a rising demand for information security in these systems. In this article, we have discussed about many file data breaches in the past and current history and they are going to increase day by day as the reports by DataLossDB (Open Security Foundation) organization, a non-profit organization in US.

#### Initial Ranging for Prioritized Network Entry in IEEE 802.16 Network. (arXiv:1312.1437v1 [cs.NI])

Prioritized network entry is desirable for establishing preferential network connectivity for the higher priority users when different priority users exist over a given network. In line with such desirability, we propose a simple but efficient priority differentiated initial ranging mechanism considering an Orthogonal Frequency Division Multiple Access (OFDMA) based IEEE 802.16 network. In the proposed mechanism, we introduce an approach that integrates an explicit CDMA-ranging code reservation scheme with a Ranging Slot Selection Window (RSSW) differentiation scheme. Simulation results are provided to characterize the performance of the proposed mechanism.

#### Computing in Operations Research using Julia. (arXiv:1312.1431v1 [math.OC])

The state of numerical computing is currently characterized by a divide between highly efficient yet typically cumbersome low-level languages such as C, C++, and Fortran and highly expressive yet typically slow high-level languages such as Python and MATLAB. This paper explores how Julia, a modern programming language for numerical computing which claims to bridge this divide by incorporating recent advances in language and compiler design (such as just-in-time compilation), can be used for implementing software and algorithms fundamental to the field of operations research, with a focus on mathematical optimization. In particular, we demonstrate algebraic modeling for linear and nonlinear optimization and a partial implementation of a practical simplex code. Extensive cross-language benchmarks suggest that Julia is capable of obtaining state-of-the-art performance.

#### Don't sit on the fence: A static analysis approach to automatic fence insertion. (arXiv:1312.1411v1 [cs.LO])

Modern architectures rely on memory fences to prevent undesired weakenings of memory consistency between threads. As both the semantics of the program under these architectures and the semantics of these fences may be subtle, the automation of their placement is highly desirable. However, precise methods to restore strong consistency do not scale to the size of deployed systems code. We choose to trade some precision for genuine scalability: we present a novel technique suitable for interprocedural analysis of large code bases. We implement this method in our new musketeer tool, and detail experiments on more than 350 executables of packages found in a Debian Linux distribution, e.g. memcached (about 10000 LoC).

#### Handling Algebraic Effects. (arXiv:1312.1399v1 [cs.LO])

Algebraic effects are computational effects that can be represented by an equational theory whose operations produce the effects at hand. The free model of this theory induces the expected computational monad for the corresponding effect. Algebraic effects include exceptions, state, nondeterminism, interactive input/output, and time, and their combinations. Exception handling, however, has so far received no algebraic treatment.

We present such a treatment, in which each handler yields a model of the theory for exceptions, and each handling construct yields the homomorphism induced by the universal property of the free model. We further generalise exception handlers to arbitrary algebraic effects. The resulting programming construct includes many previously unrelated examples from both theory and practice, including relabelling and restriction in Milner's CCS, timeout, rollback, and stream redirection.

#### A Passivity Framework for Modeling and Mitigating Wormhole Attacks on Networked Control Systems. (arXiv:1312.1397v1 [cs.SY])

Networked control systems consist of distributed sensors and actuators that communicate via a wireless network. The use of an open wireless medium and unattended deployment leaves these systems vulnerable to intelligent adversaries whose goal is to disrupt the system performance. In this paper, we study the wormhole attack on a networked control system, in which an adversary establishes a link between two distant regions of the network by using either high-gain antennas, as in the out-of-band wormhole, or colluding network nodes as in the in-band wormhole. Wormholes allow the adversary to violate the timing constraints of real-time control systems by delaying or dropping packets, and cannot be detected using cryptographic mechanisms alone. We study the impact of the wormhole attack on the network flows and delays and introduce a passivity-based control-theoretic framework for modeling the wormhole attack. We develop this framework for both the in-band and out-of-band wormhole attacks as well as complex, hereto-unreported wormhole attacks consisting of arbitrary combinations of in-and out-of band wormholes. We integrate existing mitigation strategies into our framework, and analyze the throughput, delay, and stability properties of the overall system. Through simulation study, we show that, by selectively dropping control packets, the wormhole attack can cause disturbances in the physical plant of a networked control system, and demonstrate that appropriate selection of detection parameters mitigates the disturbances due to the wormhole while satisfying the delay constraints of the physical system.

#### Behavior Modification and Utility Learning via Energy Disaggregation. (arXiv:1312.1394v1 [math.DS])

The utility company has many motivations for modifying energy consumption patterns of consumers such as revenue decoupling and demand response programs. We model the utility company--consumer interaction as a principal--agent problem. We present an iterative algorithm for designing incentives while estimating the consumer's utility function. Incentives are designed using the aggregated as well as the disaggregated (device level) consumption data. We simulate the iterative control (incentive design) and estimation (utility learning and disaggregation) process for examples including the design of incentives based on the aggregate consumption data as well as the disaggregated consumption data.

#### An Analytical Model for Loc/ID Mappings Caches. (arXiv:1312.1378v1 [cs.NI])

Concerns regarding the scalability of the inter-domain routing have encouraged researchers to start elaborating a more robust Internet architecture. While consensus on the exact form of the solution is yet to be found, the need for a semantic decoupling of a node's location and identity is generally accepted as a promising way forward. However, this typically requires the use of caches that store temporal bindings between the two namespaces, to avoid hampering router packet forwarding speeds. In this article, we propose a methodology for an analytical analysis of cache performance that relies on the working-set theory. We first identify the conditions that network traffic must comply with for the theory to be applicable and then develop a model that predicts average cache miss rates relying on easily measurable traffic parameters. We validate the result by emulation, using real packet traces collected at the egress points of a campus and an academic network. To prove its versatility, we extend the model to consider cache polluting user traffic and observe that simple, low intensity attacks drastically reduce performance, whereby manufacturers should either overprovision router memory or implement more complex cache eviction policies.

#### The Composition Theorem for Differential Privacy. (arXiv:1311.0776v2 [cs.DS] UPDATED)

Interactive querying of a database degrades the privacy level. In this paper we answer the fundamental question of characterizing the level of differential privacy degradation as a function of the number of adaptive interactions and the differential privacy levels maintained by the individual queries. Our solution is complete: the privacy degradation guarantee is true for every privacy mechanism and, further, we demonstrate a sequence of privacy mechanisms that do degrade in the characterized manner. The key innovation is the introduction of an operational interpretation (involving hypothesis testing) to differential privacy and the use of the corresponding data processing inequalities. Our result improves over the state of the art and has immediate applications to several problems studied in the literature.

#### Explicit and Implicit Constrained-Space Probabilistic Threshold Range Queries for Moving Objects. (arXiv:1311.0320v2 [cs.DB] UPDATED)

This paper studies the constrained-space probabilistic threshold range query (CSPTRQ) for moving objects. We differentiate two kinds of CSPTRQs: implicit and explicit ones. Specifically, for each moving object $o$, we assume $o$ cannot be located in some specific areas, we model its location as a closed region, $u$, together with a probability density function, and model a query range, $R$, as an arbitrary polygon. An implicit CSPTRQ can be reduced to a search (over all the $u$) that returns a set of objects, which have probabilities higher than a probability threshold $p_t$ to be located in $R$, where $0\leq p_t\leq 1$. In contrast, an explicit CSPTRQ returns a set of tuples in form of ($o$, $p$) such that $p\geq p_t$, where $p$ is the probability of $o$ being located in $R$. A straightforward adaptation of existing method is inefficient due to its weak pruning/validating capability. In order to efficiently process such queries, we propose targeted solutions, in which three main ideas are incorporated: (1) swapping the order of geometric operations based on the computation duality; (2) pruning unrelated objects in the early stages using the location unreachability; and (3) computing the probability using the multi-step mechanism. Extensive experimental results demonstrate the efficiency and effectiveness of the proposed algorithms.

#### Coloring triangle-free rectangle overlap graphs with $O(\log\log n)$ colors. (arXiv:1301.0541v2 [cs.CG] UPDATED)

Recently, Pawlik et al. have shown that triangle-free intersection graphs of line segments in the plane can have arbitrarily large chromatic number. Specifically, they construct triangle-free segment intersection graphs with chromatic number $\Theta(\log\log n)$. Essentially the same construction produces $\Theta(\log\log n)$-chromatic triangle-free intersection graphs of a variety of other geometric shapes---those belonging to any class of compact arc-connected subsets of $\mathbb{R}^2$ closed under horizontal scaling, vertical scaling, and translation, except for axis-aligned rectangles.

We show that this construction is asymptotically optimal for intersection graphs of boundaries of axis-aligned rectangles. This class of graphs can be alternatively defined as the class of overlap graphs of axis-aligned rectangles, that is, graphs in which two rectangles are connected by an edge if they intersect but are not nested. We prove that triangle-free rectangle overlap graphs have chromatic number $O(\log\log n)$, improving on the previous bound of $O(\log n)$. To this end, we exploit a relationship between off-line coloring of rectangle overlap graphs and on-line coloring of interval overlap graphs. Our coloring method decomposes the graph into a bounded number of subgraphs with a tree-like structure that "encodes" strategies of the adversary in the on-line coloring problem, and colors these subgraphs with $O(\log\log n)$ colors using a combination of techniques from on-line algorithms (first-fit) and data structure design (heavy-light decomposition).

#### A control problem with fuel constraint and Dawson-Watanabe superprocesses. (arXiv:1207.5809v7 [math.PR] UPDATED)

We solve a class of control problems with fuel constraint by means of the log-Laplace transforms of $J$-functionals of Dawson-Watanabe superprocesses. This solution is related to the superprocess solution of quasilinear parabolic PDEs with singular terminal condition. For the probabilistic verification proof, we develop sharp bounds on the blow-up behavior of log-Laplace functionals of $J$-functionals, which might be of independent interest.

#### Solving the Canonical Representation and Star System Problems for Proper Circular-Arc Graphs in Log-Space. (arXiv:1202.4406v5 [cs.CC] UPDATED)

We present a logspace algorithm that constructs a canonical intersection model for a given proper circular-arc graph, where canonical' means that models of isomorphic graphs are equal. This implies that the recognition and the isomorphism problems for this class of graphs are solvable in logspace. For a broader class of concave-round graphs, that still possess (not necessarily proper) circular-arc models, we show that those can also be constructed canonically in logspace. As a building block for these results, we show how to compute canonical models of circular-arc hypergraphs in logspace, which are also known as matrices with the circular-ones property. Finally, we consider the search version of the Star System Problem that consists in reconstructing a graph from its closed neighborhood hypergraph. We solve it in logspace for the classes of proper circular-arc, concave-round, and co-convex graphs.

#### A limit process for partial match queries in random quadtrees and $2$-d trees. (arXiv:1202.1342v5 [math.PR] UPDATED)

We consider the problem of recovering items matching a partially specified pattern in multidimensional trees (quadtrees and $k$-d trees). We assume the traditional model where the data consist of independent and uniform points in the unit square. For this model, in a structure on $n$ points, it is known that the number of nodes $C_n(\xi )$ to visit in order to report the items matching a random query $\xi$, independent and uniformly distributed on $[0,1]$, satisfies $\mathbf {E}[{C_n(\xi )}]\sim\kappa n^{\beta}$, where $\kappa$ and $\beta$ are explicit constants. We develop an approach based on the analysis of the cost $C_n(s)$ of any fixed query $s\in[0,1]$, and give precise estimates for the variance and limit distribution of the cost $C_n(x)$. Our results permit us to describe a limit process for the costs $C_n(x)$ as $x$ varies in $[0,1]$; one of the consequences is that $\mathbf {E}[{\max_{x\in[0,1]}C_n(x)}]\sim \gamma n^{\beta}$; this settles a question of Devroye [Pers. Comm., 2000].

#### Task swapping networks in distributed systems. (arXiv:1107.1851v3 [cs.DC] UPDATED)

In this paper we propose task swapping networks for task reassignments by using task swappings in distributed systems. Some classes of task reassignments are achieved by using iterative local task swappings between software agents in distributed systems. We use group-theoretic methods to find a minimum-length sequence of adjacent task swappings needed from a source task assignment to a target task assignment in a task swapping network of several well-known topologies.

#### Selfishness Level of Strategic Games. (arXiv:1105.2432v6 [cs.GT] UPDATED)

We introduce a new measure of the discrepancy in strategic games between the social welfare in a Nash equilibrium and in a social optimum, that we call \bfe{selfishness level}. It is the smallest fraction of the social welfare that needs to be offered to each player to achieve that a social optimum is realized in a pure Nash equilibrium. The selfishness level is unrelated to the price of stability and the price of anarchy and is invariant under positive linear transformations of the payoff functions. Also, it naturally applies to other solution concepts and other forms of games.

We study the selfishness level of several well-known strategic games. This allows us to quantify the implicit tension within a game between players' individual interests and the impact of their decisions on the society as a whole. Our analyses reveal that the selfishness level often provides a deeper understanding of the characteristics of the underlying game that influence the players' willingness to cooperate.

In particular, the selfishness level of finite ordinal potential games is finite, while that of weakly acyclic games can be infinite. We derive explicit bounds on the selfishness level of fair cost sharing games and linear congestion games, which depend on specific parameters of the underlying game but are independent of the number of players. Further, we show that the selfishness level of the $n$-players Prisoner's Dilemma is $1/(2n-3)$, that of the $n$-players public goods game is $(1-\frac{c}{n})/(c-1)$, where $c$ is the public good multiplier, and that of the Traveler's Dilemma game is $\frac12$. Finally, the selfishness level of Cournot competition (an example of an infinite ordinal potential game), Tragedy of the Commons, and Bertrand competition is infinite.

#### P != NP Proof. (arXiv:0907.3965v70 [cs.CC] UPDATED)

This paper demonstrates that P \not= NP. The way was to generalize the traditional definitions of the classes P and NP, to construct an artificial problem (a generalization to SAT: The XG-SAT, much more difficult than the former) and then to demonstrate that it is in NP but not in P (where the classes P and NP are generalized and called too simply P and NP in this paper, and then it is explained why the traditional classes P and NP should be fixed and replaced by these generalized ones into Theory of Computer Science). The demonstration consists of: 1. Definition of Restricted Type X Program; 2. Definition of the General Extended Problem of Satisfiability of a Boolean Formula - XG-SAT; 3. Generalization to classes P and NP; 4. Demonstration that the XG-SAT is in NP; 5. Demonstration that the XG-SAT is not in P; 6. Demonstration that the Baker-Gill-Solovay Theorem does not refute the proof; 7. Demonstration that the Razborov-Rudich Theorem does not refute the proof; 8. Demonstration that the Aaronson-Wigderson Theorem does not refute the proof.

### StackOverflow

#### Get PID of a child process after finishing in FreeBSD and C language

I'm trying to handle SIGCHLD properly, but I don't manage to get the PID of the process inside the handler so I can change the value of one parameter inside an structure.

Here is the code:

typedef struct
{
int active_state;
int pid;
}Worker;

typedef struct
{
Worker worker;
int used_state;//this can be USED or UNUSED
}WorkersTable;

WorkersTable *table; //I put this in a shared memory map and works correctly


Here is the code for the handler.Inside this file there is a global variable called dead_child_pid that I want to to to store the pid of the dead child to be used alter.

void handler_SIGCHLD(int signal)
{
pid_t child_pid;
int e;
do
{
child_pid=wait3(&e,WNOHANG,NULL);
}while(child_pid>(pid_t)0);
mark_unused=1;
}


When handler_SIGCHLD is called and at the end we I put mark_unused=1, the following code is accessed:

if(mark_unused)
{
/*put the current position at the table as unused*/
int counter_find_pid=0;
{
counter_find_pid++;
}
table[counter_find_pid].used_state=UNUSED;
}


### QuantOverflow

#### Garch modelling on Stata

I would like to ask "how to do GARCH modelling on stata".

Basically I want to estimate stock market volatility using daily data. I have one variable as return series, $r_t=\ln(\frac{P_t}{P_{t-1}})$.

I need a step by step explanation.

### StackOverflow

#### Setting variable to default value

There's a class that uses an ArrayBuffer and I don't really care what is the value of the first element. In C# there's default keyword that returns null for reference types and default value (e.g. 0 for int) for value types. How to achieve something similar in Scala? I don't want to use Option because then there would be boilerplate code to extract values.

#### Creating a Java Enum in Scala

My workplace has been experimenting in moving from Java to Scala for some tasks, and it works well for what we're doing. However, some preexisting logging methods expect a java.lang.Enum. The logging method is defined in the (Java) base class, and the subclasses can define their own enums, which the logger will track across all instances in multiple threads/machines.

It works like this in Java:

public class JavaSubClass extends JavaBaseClass {
enum Counters {
GOOD_THING
}

// some work here
if(terribleThing) {
} else {
loggingMethod(Counters.GOOD_THING)
// more work here
}
}
}


Then, when the task has finished, we can see that

BAD_THING: 230
GOOD_THING: 10345


Is there any way to replicate this in Scala, either by creating Java Enums or converting from Enumeration to Enum? I have tried extending Enum directly, but it seems to be sealed, as I get the error in the console:

error: constructor Enum in class Enum cannot be accessed in object $iw Access to protected constructor Enum not permitted because enclosing object$iw is not a subclass of
class Enum in package lang where target is defined


### CompsciOverflow

#### Is there a DFA with $k+2$ states which its reverse has $2^k$ states

I am trying to figure out if there exists a DFA $M$ with $k+2$ states (for every $k\in \mathbb{N}$ ) so that every automaton which accepts $L(M)^R$ has at least $2^k$ states.
I am trying to find an example of such a DFA, any help?

#### Length of a two-state unifying word

This question popped up in my mind after reading about DFA's. If we have a DFA $M$ with $n$ states, and we know that a word $w$ (not necessarily in $L(M)$) that unifies two words, what can we say about its' length?

A word $w$ unifies two states $q_1, q_2 \in Q$ if there is a $q' \in Q$ such that $\delta(q_1, w) = q'$ and $\delta(q_2, w) = q'$.

### StackOverflow

#### How can I look up a spray-json formatter by the runtime type of a value?

The traditional way to use spray-json seems to be to bind all your models to appropriate JsonFormats (built-in or custom) at compile time, with the formats all as implicits. Is there a way instead to look the formatters up at runtime? I'm trying to marshal a heterogeneous list of values, and the only ways I'm seeing to do it are

1. Write an explicit lookup (e.g. using pattern matching) that hard-codes which fomratter to use for which value type, or
2. Something insane using reflection to find all the implicits

I'm pretty new to Scala and spray-json both, so I worry I'm missing some simpler approach.

More context: I'm trying to write a custom serializer that writes out only a specified subset of (lazy) object fields. I'm looping over the list of specified fields (field names) at runtime and getting the values by reflection (actually it's more complicated than that, but close enough), and now for each one I need to find a JsonFormat that can serialize it.

### StackOverflow

#### play scala: How to write file in deployment?

I'am writting some files on my dev using :

new java.io.File("/public/reports/test.txt")


And it's working file But in production it's not working. My guess is because in production the project is in a jar (http://www.playframework.com/documentation/2.0.x/ProductionDist )

Is they a way to write into that jar?

Thanks

#### play-slick and Oracle driver

Following this question I override the Config object and provide additional drivers.

However, the CurrentDBAction (which extends PredicatedDBAction) call the DB object from slick package. In this object we are finally calling the Database apply function which read the configuration from the slick Config object. Damned !

Is there a way to override the map drivers without redifining too much objects ? some Scala magic perhaps ?

thanks a lot.

### CompsciOverflow

#### Is an $\mathcal{O}(n\times \text{Number of clusters})$ clustering algorithm useful?

I am a physicist, with little formal training in computer science - please don't assume I know even obvious things about computer science!

Within the context of data analysis, I was interested in identifying clusters within a $d$-dimensional list of $n$ data-points, for which the dimensionality $d$ could be $\sim100$, whilst the number of data-points could be $\sim 1,000,000$, or perhaps more.

I wanted the points with a cluster to be close together, with distance measured in the Euclidean manner, $$d(\vec x,\vec y) = \sqrt{\sum_{i=1}^d (x_i - y_i)^2}$$ As long as the clustering was reasonably accurate, I wasn't bothered if the exactly correct result was obtained. i.e. if of my $\sim1,000,000$, $\sim1,000$ points were wrongly categorized, it wouldn't matter much.

I have written a short algorithm that can perform typically at $\mathcal{O}(n)$ (from trials of up to $n\sim5,000,000$ and some theoretical analysis) and worst-case $\mathcal{O}(n^2)$ (from my theoretical evaluation of the algorithm). The nature of the algorithm sometimes (but not always) avoids the so-called chaining problem in clustering, where dissimilar clusters are chained together because they have a few data-points that are close.

The complexity is, however, sensitive to the a priori unknown number of clusters in the data-set. The typical complexity is, in fact, $\mathcal{O}(n\times c)$, with $c$ the number of clusters.

Is that better than currently published algorithms? I know naively it is a $\mathcal{O}(n^3)$ problem. I have read of SLINK, that optimizes the complexitiy to $\mathcal{O}(n^2)$. If so, is my algorithm useful? Or do the major uses of clustering algorithms require exact solutions?

In real applications is $c\propto n$?, such that my algorithm has no advantage. My naive feeling is that for real problems, the number of "interesting" clusters (i.e. not including noise) is a property of the physical system/situation being investigated, and is in fact a constant, with no significant dependence on $n$, in which case my algorithm looks useful.

#### SQL statements - without group by [on hold]

If I wanted to find all values in a table that occur more than twice without using group by, how would I do that?

For example, if I had last names in a certain zip code, and I wanted to find entries with this last name more than twice, how would I do this without group by in SQL statements?

I tried

select name, count() from population order by name asc having count() > 2;

but that doesn't do what I want it to. Any suggestions?

### StackOverflow

#### Is there a strategy to test macros in Clojure?

I would generally make functions and would write unit tests for them. but for macros that do code transformations, how will one apply logic of macthing expected and actual value ?

because input to macros will be code and output will be code as well. this seems so dynamic to me to be able to come up with some strategy to test.

Or I can use a sample input for macro, and pass the expression to macroexpand-1 and match the output with the expected one ?

#### Why does building a release APK fail only when android:debuggable="false" is set?

Recently, the Google Play Store seems to be enforcing that all uploaded apps must set android:debuggable="false" in the AndroidManifest.xml file. I previously had android:debuggable="true" set.

However, when I set android:debuggable="false" my ant clean release blows up in the -obfuscate: step.

ProGuard seems to be the culprit:

 /usr/local/Cellar/android-sdk/r21.1/tools/ant/build.xml:875: Can't write [/Users/-/Code/android/bin/proguard/obfuscated.jar] (Can't read [/Users/-/Code/android/libs/android-async-http-1.4.3-no-redirect.jar] (Duplicate zip entry [com/a/a/a/b.class == android-async-http-1.4.3-no-redirect.jar:com/loopj/android/http/AsyncHttpClient$1.class]))  Any guidance on how to proceed? What changes in the obsfuscate step when doing a release build with android:debuggable="true" vs a release build with android:debuggable="false"? Some additional details: • doing a standard ant clean release • project is actually written in Scala • using the level 17 build tools • ProGuard 4.4 ### /r/compsci #### Math Majors for CS?? I am a first year undergrad and am very much interested in computer science. But my college which is a grad school doesnot provide a major in compsci(as the undergrad program has recently started). So I'll have to take a math major instead. But luckily there are electives like graph theory and combinatorics, Datastructs and algo, Automata theory , Design and analysis of algo,game theory ,scientific computing, algorithmic algebra. Will all these courses along with my math major , be fruitful in my research career in compsci. Will I be able to land with good jobs in IT giants?? submitted by Archillezzz [link] [16 comments] ### StackOverflow #### Clojure Leining REPL OutOfMemoryError Java heap space I am trying to parse a fairly small (< 100MB) xml file with: (require '[clojure.data.xml :as xml] '[clojure.java.io :as io]) (xml/parse (io/reader "data/small-sample.xml"))  and I am getting an error: OutOfMemoryError Java heap space clojure.lang.Numbers.byte_array (Numbers.java:1216) clojure.tools.nrepl.bencode/read-bytes (bencode.clj:101) clojure.tools.nrepl.bencode/read-netstring* (bencode.clj:153) clojure.tools.nrepl.bencode/read-token (bencode.clj:244) clojure.tools.nrepl.bencode/read-bencode (bencode.clj:254) clojure.tools.nrepl.bencode/token-seq/fn--3178 (bencode.clj:295) clojure.core/repeatedly/fn--4705 (core.clj:4642) clojure.lang.LazySeq.sval (LazySeq.java:42) clojure.lang.LazySeq.seq (LazySeq.java:60) clojure.lang.RT.seq (RT.java:484) clojure.core/seq (core.clj:133) clojure.core/take-while/fn--4236 (core.clj:2564)  Here is my project.clj: (defproject dats "0.1.0-SNAPSHOT" ... :dependencies [[org.clojure/clojure "1.5.1"] [org.clojure/data.xml "0.0.7"] [criterium "0.4.1"]] :jvm-opts ["-Xmx1g"])  I tried setting a LEIN_JVM_OPTS and JVM_OPTS in my .bash_profile without success. When I tried the following project.clj: (defproject barber "0.1.0-SNAPSHOT" ... :dependencies [[org.clojure/clojure "1.5.1"] [org.clojure/data.xml "0.0.7"] [criterium "0.4.1"]] :jvm-opts ["-Xms128m"])  I get the following error: Error occurred during initialization of VM Incompatible minimum and maximum heap sizes specified Exception in thread "Thread-5" clojure.lang.ExceptionInfo: Subprocess failed {:exit-code 1}  Any idea how I could increase the heap size for my leiningen repl? Thanks. ### CompsciOverflow #### Prove the existence of regular$C$so that:$A \prec C \prec B $Given$A,B$regular languages with$A \prec B$. Prove the existence of$C\in L_{\text{regular}}$so that:$A \prec C \prec B$. Here,$A\prec B$stands for:$A\subset B $and$B\setminus A $is infinite. I tried to go for:$C=\overline{B} \cup A$and some other options but it didn't work out. ### StackOverflow #### Evaluate expression according to the first true condition In javascript we can do things like this:  switch (true) { case a > b: expression; break; case a < b: expression; break; case a == b: expression; break; }  Can we achieve something like this in Scala? If expression is not the case. Maximum that I could figure out is: val a = foo < elem val b = foo > elem val c = foo == elem true match { case a => case b => case c => }  ### Lobsters #### Intel i7 loop performance anomaly ### HN Daily #### Daily Hacker News for 2013-12-05 ### Planet Theory #### The Parameterized Complexity of Reasoning Problems Beyond NP Authors: Ronald de Haan, Stefan Szeider Download: PDF Abstract: Today's propositional satisfiability (SAT) solvers are extremely powerful and can be used as an efficient back-end for solving NP-complete problems. However, many fundamental problems in knowledge representation and reasoning are located at the second level of the Polynomial Hierarchy or even higher, and hence polynomial-time transformations to SAT are not possible, unless the hierarchy collapses. Recent research shows that in certain cases one can break through these complexity barriers by fixed-parameter tractable (fpt) reductions which exploit structural aspects of problem instances in terms of problem parameters. In this paper we develop a general theoretical framework that supports the classification of parameterized problems on whether they admit such an fpt-reduction to SAT or not. This framework is based on several new parameterized complexity classes. We underpin the robustness of our theory by providing a characterization of the new complexity classes in terms of weighted QBF satisfiability, alternating Turing machines, and first-order model checking. We instantiate our theory by classifying the complexities of several problems with respect to various natural parameters, including the consistency problem for disjunctive answer set programming, a robust variant of constraint satisfaction, the problem of minimizing DNF formulas and implicant cores of DNF formulas, a problem related to extending cliques in graphs, and variants of Fagin-definable problems. #### Vertex Disjoint Path in Upward Planar Graphs Authors: Saeed Amiri, Ali Golshani, Stephan Kreutzer, Sebastian Siebertz Download: PDF Abstract: The$k$-vertex disjoint paths problem is one of the most studied problems in algorithmic graph theory. In 1994, Schrijver proved that the problem can be solved in polynomial time for every fixed$k$when restricted to the class of planar digraphs and it was a long standing open question whether it is fixed-parameter tractable (with respect to parameter$k$) on this restricted class. Only recently, \cite{CMPP}.\ achieved a major breakthrough and answered the question positively. Despite the importance of this result (and the brilliance of their proof), it is of rather theoretical importance. Their proof technique is both technically extremely involved and also has at least double exponential parameter dependence. Thus, it seems unrealistic that the algorithm could actually be implemented. In this paper, therefore, we study a smaller class of planar digraphs, the class of upward planar digraphs, a well studied class of planar graphs which can be drawn in a plane such that all edges are drawn upwards. We show that on the class of upward planar digraphs the problem (i) remains NP-complete and (ii) the problem is fixed-parameter tractable. While membership in FPT follows immediately from \cite{CMPP}'s general result, our algorithm has only single exponential parameter dependency compared to the double exponential parameter dependence for general planar digraphs. Furthermore, our algorithm can easily be implemented, in contrast to the algorithm in \cite{CMPP}. ## December 05, 2013 ### StackOverflow #### Better way of creating a flat list out of numbers and vectors I've got a function like this: (defn magic [a b c] (flatten (conj [] a b c)))  So on these inputs I get the following: (magic 1 2 3) => (1 2 3) (magic 1 [2 3] 4) => (1 2 3 4)  My question is, is there a better way of doing this? The problem can be summarised as: I don't know whether I will get numbers or vectors as input, but I need to return a single flat list #### Akka FSM actor and round-robin routing I want to convert some set of actors into FSM using Akka FSM. Currently system is designed in the way that every actor knows what to do with results of it's action and which actor is next in sequence of processing. Now I want to have some sort of dedicated actors, which are doing only things they should know (and now know about entire message routing), and central FSM, which knows how to route messages and process transformation flow. Client sends some request to FSM actor, FSM actor - on transition to next state - sends message to some actor in onTransition block. That actor replies to sender with some message, which is processed inside FSM state somehow until request is finished. So far everything looks good, however I'm not sure what will happen if multiple clients will start interaction with FSM actor. Will the "workflow" be recorded somewhere, so flows from different clients won't collide at some point (like, FSM actor receives message from another client instead of originating one)? Is it safe to have say 10 FSM actors and round-robin router, or I need to create new FSM actor on every request from client, and then kill it once finished? #### SBT Publish local does not include dependencies I have multiple projects A, B, C. Some are utility libraries and some are applications. If I have the dependency heirarchy A->B->C and use publish-local I have to include Both dependency B and C in A. Even if A does not use anything directly from C (only by proxy). It is as if all project B dependencies are added with provided scope. How come these dependencies won't be resolved and added to the classpath? This is not a multi project Build.scala (on purpose) I have several simplified build files and I want them to be self contained build-wise. I'm using the command re-start (Revolver) when starting the application A. /Magnus ### CompsciOverflow #### Converting English into Logic I'm learning about Logic; I've devised a few sentences to convert into logic to test myself and see if i've grasped the topic. It would be of great help if you could tell me if i'm doing it right! • Every student who likes chocolate is smart: (Every student) (who likes chocolate) (is smart) ∀x ∈D, [Like (Student (x), chocolate) → Smart(x)] • There exists a barber that shaves all men who do not shave themselves. (There exists a barber) (that shaves all men) (if and only if they do not shave themselves) ∃x ∈D, [Shaves (x, ∀y) → Shave (y, y)] Thank you ### /r/scala #### Strings are untyped - using Scalaz's type tags ### StackOverflow #### What does Clojure borrow from Haskell? In code quarterly interview with Rich Hickey Michael Fogus hints that Clojure borrows a lot from Haskell: Fogus: As someone who only knows Haskell enough to read the papers, Clojure appears to be influenced by it substantially. From the names and operation of core functions—take, drop, iterate, repeat, etc.—to its protocols facility, there is a lot in Clojure that a Haskell programmer would recognize. Can you elaborate on Haskell’s influences on Clojure both positive and negative? Hickey: I think Haskell is a fantastic, awe-inspiring piece of work. I haven’t used it in anger, but it certainly was a positive influence. Haskell obviously goes much further than Clojure in pursuing the ideals of functional programing. In particular they differ in the approach to using types to enforce things. My question is - what parts of Clojure are influenced by Haskell? Fogus hints at some syntax, and there is obviously functional laziness. What about the STM and concurrency operators? Assumptions: • I'm asking this question in a positive tone. I think it is great that Rich could see some of the treasures in Haskell and use them. ### CompsciOverflow #### What is the complexity of this bubble sort algorithm? [duplicate] This question already has an answer here: I have been doing a little reading up on bubble sort and have read on wikipedia that it's complexity is measured as$\Theta(n^2)$This bubble sort however is slightly more efficient. I thought this would be the best place to ask how I would work out this particular implementations complexity seeing that the number of iterations in the inner loop is reduced with each pass. for (top = items.Length; top > 0; top--) { for (low = 0, high = 1; high < top; low++, high++) { if (items[low].CompareTo(items[high]) > 0) { tmp = items[high]; items[high] = items[low]; items[low] = tmp; } } }  ### Fefe #### Tolle Idee: Bundeswehrsoldaten sollen jetzt Asylanträge ... Tolle Idee: Bundeswehrsoldaten sollen jetzt Asylanträge bearbeiten. Was kann da schon schiefgehen! (via) ### /r/netsec #### HTTP verb tampering demo [YouTube 6:35] ### StackOverflow #### Serialisation and Deserialisation over clrmq I am currently in the process of designing a networking wrapper in C# using the clrmq binding for ZeroMQ. My problem is that I am trying to think of a way to do deserialization and serialization transparently; that is, the networking wrapper can transmit any object as bytes without knowing or caring about its type and can deserialize to the correct type on the other end. I have a class heirachy as follows public interface ITransmission { Type Type { get; } } public abstract class Transmission<T> : ITransmission { public Type Type { get { return typeof(T); } } }  The idea is to serialize objects that have implemented the Transmission type;transmit them over clrmq as bytes and deserialize them at the other end My ideal solution would 1. Be capable of serializing any type of object with no attribute decoration or very minimal attribute decoration to preferably a byte array format (hence the idea above using the Transmission object) 2. Be able to deserialize the received byte array format to a typed object What are people currently using or doing to resolve the issue of serializing and deserialising transparently? Speed will be a factor at some point but at this time , I'm just trying to find out what the options are. Is the implied limitation here that the networking libraries will need to have references to all the assemblies containing the types to be transmitted? Additionally, the other question would be whether this a feasible approach to take, or should a server and client be strongly typed from the get go? Thanks for reading. #### Creating a distributed memory service in Scala How do you think Scala can help us to write a distributed memory service like Redis (but multi-thread)? Scala programming encourages us to use immutable data in order to avoid shared data and race conditions, however in this kind of data storage immutable data can't help us (I think). Blocking threads would be the solution? ### /r/netsec #### Expressing Yourself: Analysis of a .Net Elevation of Privilege Vulnerability submitted by _rs [link] [comment] ### StackOverflow #### could not find implicit value for parameter marshaller: spray.httpx.marshalling.ToResponseMarshaller I'm using val akkaV = "2.2.3" val sprayV = "1.2.0" Seq( "io.spray" % "spray-can" % sprayV, "io.spray" % "spray-routing" % sprayV, "io.spray" %% "spray-json" % "1.2.5", "io.spray" % "spray-testkit" % sprayV, "com.typesafe.akka" %% "akka-actor" % akkaV, "com.typesafe.akka" %% "akka-testkit" % akkaV,  And getting this error: could not find implicit value for parameter marshaller: spray.httpx.marshalling.ToResponseMarshaller[List[org.bwi.models.Cluster]] with this code: object JsonImplicits extends DefaultJsonProtocol { val impCluster = jsonFormat2(Cluster) } trait ToolsService extends HttpService with spray.httpx.SprayJsonSupport { val myRoute = { import JsonImplicits._ path("") { get { getFromResource("tools.html") } } ~ pathPrefix("css") { get { getFromResourceDirectory("css") } } ~ pathPrefix("fonts") { get { getFromResourceDirectory("fonts") } } ~ pathPrefix("js") { get { getFromResourceDirectory("js") } } ~ path("clusters") { get { complete { val result: List[Cluster] = List(Cluster("1", "1 d"), Cluster("2", "2 d"), Cluster("3", "3 d")) result //***** ERROR OCCURS HERE ***** } } } }  } I've tried the suggestion on this question but it did not work, same error. I can't seem to figure out what the implicit I need to import is. Any help would be appreciated. ### Planet Clojure #### SICP in Clojure – Update In my post, SICP in Clojure, I incorrectly identified Steve Deobald as the maintainer of this project. The original maintainer of the project placed a link on the site saying that Steve is the maintainer. That is not correct. Apologies to Steve and apologies to my readers who were hopeful this project would be going forward. Any thoughts on moving this project forward? I think the idea is a very sound one. PS: Unlike many media outlets, I think corrections should be as prominent as the original mistakes. ### StackOverflow #### Accessing the index of a particular cell in Scala I have to write a method "all()" which returns a list of tuples; each tuple will contain the row, column and set relevant to a particular given row and column, when the function meets a 0 in the list. I already have written the "hyp" function which returns the set part I need, eg: Set(1,2). I am using a list of lists:  | 0 | 0 | 9 | | 0 | x | 0 | | 7 | 0 | 8 |  If Set (1,2) are referring to the cell marked as x, all() should return: (1,1, Set(1,2)) where 1,1 are the index of the row and column. ### CompsciOverflow #### In basic terms, what is the definition of P, NP, NP-Complete, and NP-Hard? I'm in a course about computing and complexity, and am unable to understand what these terms mean. All I know is that np is a subset of np complete which is a subset of np hard... but I have no idea what they actually mean. Wikipedia isn't much help either as the explanations are still a bit too high level. ### QuantOverflow #### Can't understand arbitrage in B-S model. Please Help! I'm trying to understand the derivation of Black-Scholes PDE, but I find a spot I really cannot follow (I'm following the original bond replication method proposed in B-S paper). At some point, once the Delta-hedging is made, the portfolio becomes riskless, and the argument follows "since it is riskless, it must earn the risk-free interest rate, because otherwise arbitrage opportunities would arise." The problem I find is that we do NOT know ex-ante what will the return of the portfolio be, so, in order to arbitrage, we do not know if we should go long on it or short, so the arbitrage is not such... We know the value of the portfolio at present time, and we know that its return is certain, but we cannot know what will that return be -whether more or less than the free interest rate-, so we cannot arbitrage! What is that I don't get? I hope I have explained myself clearly... Any help will be much appreciated. Thanks a lot. Cheers! ### StackOverflow #### run Scala code in package (LinkedIn Norbert) I am trying to run a Scala object called NorbertClusterClientMain which is in package com.linkedin.norbert.cluster. The source code for it is in folder examples/src/main/scala of rhavyn's open-source branch of LinkedIn Norbert, and I am working on a Linux command line. Although I've been told that running Scala code in a package is like running Java in a package, I am in examples/src/main/scala but cannot use this command: $ scala com.linkedin.norbert.cluster.NorbertClusterClientMain


I am getting "No such file or class on classpath", even though the file exists.

I was successfully able to compile Norbert with

$mvn clean -DskipTests install  How can I run the NorbertClusterClientMain? Please let me know. I appreciate your help. ### Lobsters #### China Bans Financial Companies From Bitcoin Transactions ### StackOverflow #### Running scalatest classes from sbt file I am running my test class from sbt file using task scalaTaskRun := { val test = (runMain in Compile).fullInput(" org.scalatest.tools.Runner -s package.tests.TestClass1 -h ReportOutput").evaluated }  to get a html report output for single test class. But I don't want to add -s TestClass2 again for running one more tests and so on... If I want to run many test classes from sbt file just like running testNG suite xml which contains more than one test classes. How can the same thing be achieved in sbt scalatest?? I tried running with the runpath command... Runner -R target\\folder\\classes -w package.testcases -h reportFolder But it's not running the compiled test classes from classes folder. Kindly help in fixing it. #### How can I restrict access to a specific play route I'm working on an administrator area as a subproject of my main project. In my build file I have val main = play.Project(appName, appVersion, appDependencies).dependsOn(admin)  and in my routes I have -> /admin admin.Routes  The admin area works fine, but I want to be able to restrict access to those users which have the role Admin. I tried this Global  @Override public Handler onRouteRequest(Http.RequestHeader request) { if(UserInfo.getCurrent().hasRole(SecurityRole.Type.ADMIN)){ if(request.path().startsWith("/admin")){ return admin.Routes.routes().apply(Http.Context.current()._requestHeader()); } } return super.onRouteRequest(request); } }  which grabs the user from the current context. But I get the error RuntimeException: There is no HTTP Context available from here.  I suppose I could set up security in the admin area using Deadbolt, but I figured since the main file routes to the subproject there would be some easy way to just restrict access from the main project, but I haven't found anything. Any help would be appreciated. #### Dynamic orderBy with Squeryl I can not figure out how to change orderBy dynamically in runtime. I need something like: def samplesSorted(fields: List[String]) = { from(Schema.samples)(s => select(s) orderBy(fields.map(getterByName)) }  Or something like def samplesSorted(fields: List[String]) = { val q = from(Schema.samples)(s => select(s)) fields.forEach(field => q.addOrderBy(getterByName(field))) q }  I am trying to write a help function to manipulate AST now. But that does not seem like the right solution. ### Halfbakery #### Sharing Point (0.5) ### StackOverflow #### Specs2 and Mockito ordering of expected calls I am trying to verify that a few methods on a mock were called in specs2 with mockito. Some of them need to be ordered in a specific way, some others just need to be called, order does not matter. How do I do this? I have this:  there was one(db).getLinkedTeamIndex(uberName, gameId) andThen one(db).getImpostingPlayerLinkIdFor("XX", teamId, gameId) got { one(db).getLinkInCurrentTeam(wrongTeamId, gameId, uberName) one(db).switchPlayersBetweenLinks(targetTeamLinkId, wrongTeamLinkId) }  So for a while I thought this worked, as the test passes just fine. However then I realized that there in fact is no call to getImpostingPlayerLinkIdFor with an XX parameter. So the test should fail. I thought that maybe this is somehow related to the error throwing behavior, but I am using a mutuable spec and I mixed in the ThrownExpectations trait just to make sure. The test still passes. How can I verify that some methods are called in order and some other methods are called in any order? #### scalac finds wrong forAll method in ScalaTest test I have a ScalaTest 2 class that extends GeneratorDrivenPropertyChecks, and also indirectly extends FeatureSpec and Matchers (via a trait I wrote which extends those two classes). It has code like this in it: forAll(mySequence) { myItem => myItem.applicationID should be (foo.applicationID) }  This fails to compile because scalac says: [error] APISpec.scala:253: value applicationID is not a member of Seq[com.company.Item] [error] myItem.applicationID should be (foo.applicationID) [error] ^  It turns out that, at least according to Eclipse Scala IDE, the compiler is resolving "forAll" as meaning this method, in GeneratorDrivenpropertyChecks:  /** * Performs a property check by applying the specified property check function to arguments * supplied by the specified generators. * * <p> * Here's an example: * </p> * * <pre class="stHighlight"> * import org.scalacheck.Gen * * // Define your own string generator: * val famousLastWords = for { * s <- Gen.oneOf("the", "program", "compiles", "therefore", "it", "should", "work") * } yield s * * forAll (famousLastWords) { (a: String) => * a.length should equal ((a).length) * } * </pre> * * @param fun the property check function to apply to the generated arguments */ def forAll[A](genA: Gen[A], configParams: PropertyCheckConfigParam*)(fun: (A) => Unit) (implicit config: PropertyCheckConfig, shrA: Shrink[A] ) { // body omitted }  which is not the forAll method I'm wanting to use here! Is this a bug in ScalaTest (i.e. that the two methods should not both be named forAll)? And how should I call the correct method? ### Lobsters #### Extracting Immutable Objects ### StackOverflow #### What is the best functional language there today [on hold] Thank you for considering my question. I want to learn a functional language. Which is best? #### Accessing Camel/Jetty instance under akka-camel actor I have a basic Jetty instance using the akka-camel library with akka and scala. ex: class RestActor extends Actor with Consumer { def endpointUri = "jetty:https://0.0.0.0:443/" def receive = { case msg: CamelMessage => { ... } ... } ... }  I want to bind a SslSocketConnector to the underlying jetty instance so that I can load my SSL cert, but the problem I have is I can't figure out how to get a handle the underlying jetty instance to call the addConnector() function. One thought I had was to add the Camel trait to my class (ex: RestActor extends Actor with Consumer with Camel) to access Camel's "context" variable, but the compiler complains that doing this will override the "context" variable which also exists from Akka's actor trait. #### Jetty works for HTTP but not HTTPS I am trying to create a jetty consumer. I am able to get it successfully running using the endpoint uri: jetty:http://0.0.0.0:8080  However, when I modify the endpoint uri for https: jetty:https://0.0.0.0:8443  The page times out trying to load. This seems odd because the camel documentation states it should function right out of the box. I have since loaded a signed SSL into java's default keystore, with my attempted implementation to load it below:http://camel.apache.org/jetty.html I have a basic Jetty instance using the akka-camel library with akka and scala. ex: class RestActor extends Actor with Consumer { val ksp: KeyStoreParameters = new KeyStoreParameters(); ksp.setPassword("..."); val kmp: KeyManagersParameters = new KeyManagersParameters(); kmp.setKeyStore(ksp); val scp: SSLContextParameters = new SSLContextParameters(); scp.setKeyManagers(kmp); val jettyComponent: JettyHttpComponent = CamelExtension(context.system).context.getComponent("jetty", classOf[JettyHttpComponent]) jettyComponent.setSslContextParameters(scp); def endpointUri = "jetty:https://0.0.0.0:8443/" def receive = { case msg: CamelMessage => { ... } ... } ... }  This resulted in some progress, because the page does not timeout anymore, but instead gives a "The connection was interrupted" error. I am not sure where to go from here because camel is not throwing any Exceptions, but rather failing silently somewhere (apparently). Does anybody know what would cause this behavior? #### How can I create a program in Java that is able to alter its own source code, or a mathematical function that it is then to run? I want to create a program that uses an stochastic optimization algorithm to evolve mathematical functions, taking two constant numbers as inputs and evolving a function in order to produce a specific answer from those two numbers. So say the inputs are 4 and 7, and I want the answer to be 230, the program would generate a function that produces this answer from the original inputs. I'm capable of producing evolutionary algorithms, but I'm just not sure how to store the function information, randomly mutate it, and also allow that mutated function to be executed by the java program. Is this what is called meta programming? Is it possible in java? I know I can store constants that might be used in the function as variables. Thats easy. But how could I store operators, and functions such as sine cosine and tan, as variables that can be changed? ### TheoryOverflow #### Partitioning a matrix into equal-sized regions: finding the maximum I am facing the following research problem. We are given a matrix$M[1..p,1..p]$of elements such that: • each element has value in the range$[0, \frac m j]$,$j <= p$• the sum of all elements in the matrix is$m$. The matrix is partitioned into$j$axis-parallel non-overlapping rectangles. Each matrix cell with positive value is covered by exactly one rectangle. Whereas, each matrix cell with value 0 is covered by at most one rectangle. For each region, we sum the values inside it and obtain$subset\_sum_i$,$i = 1..j$. The objective is to minimize the maximum$subset\_sum_i$. The problem is to find an upper bound on$subset\_sum_i$. I do not need an algorithm for actual partitioning (I already have one). I appreciate any help! Many thanks! ### UnixOverflow #### How do I get systemd to start my ZFS mount script before doing anything else? Right now I'm using zfsonlinux on Fedora 19, and it works excellently. Mounting works at bootup as planned using the initscript (included with zfs) that systemd calls, but there's a pretty significant flaw: ZFS is mounted way, WAY too late during boot, to the point that transmission and other services get very lost without /zfs mounted and can't recover, hence failing them. What I'm after is a way to get systemd to fire up the script before doing anything else. I know how to do it using initscripts (just set it to a lower runlevel, say 2), but I haven't the foggiest when it comes to doing it with systemd's target system. I've tried setting things in the services that require zfs to be up, but it doesn't work, barring Type=idle (but there must be a better way of doing it regardless): Requires=network.target After=zfs Type=idle  And for reference's sake, here's the initscript provided by Fedora. Due to the nature of how ZFS mounts on linux (?), I can't just plop an entry into fstab and be done with it, it has to be mounted using commands. ### StackOverflow #### Imperative Language and functional programming language [on hold] Can anyone help me in briefing this " What features would need to be added for imperative language to make it genuinely useful as a functional language." I have idea on imperative language and functional language but i cannot determine exactly what features need to be added for imperative language to make it usefual as functional language ### /r/netsec #### Critical Watch Learning - BadBIOS: Weapons Grade Malware ### Halfbakery #### Monkey Computer (1.0) ### Portland Pattern Repository #### The Martian And The Apple (by 146.233.0.201 8 hours ago) ### /r/netsec #### Transport Level Security – Part 2 - SSL ### CompsciOverflow #### 3-sat to 2-sat reduction It is known that 3-SAT belong to - NP-Complete complexity problems, while 2-SAT belong to P as there is known polynomial solution to it. So you can state that there is no such reduction from 3-SAT to 2-SAT unless$P=NP$. I am looking for strong proof for this state, regardless NP belong to P or not. ### /r/compsci #### Could you fill out a quick survey for my course project? Hi /r/compsci, I have a small questionnaire (10 questions) for a course project. The purpose of this questionnaire is to help look at some things related to understanding pointers (mainly syntax). If you have even just a basic (and I mean basic) understanding of how pointers work (in C) I would really appreciate it if you took some time to answer these questions. The results would hopefully contribute towards helping people help people understand pointers better. (Since I didn't get many responses from /r/learnprogramming yesterday I thought I'd try here as well. I'm also open to suggestions on where it would be appropriate to post this, if this is not the place. /r/CSEducation looks relatively dead unfortunately.) Feedback, additional thoughts and questions are welcome. Thank You! submitted by nithokian [link] [6 comments] ### StackOverflow #### Scala Tutorial not rendered correctly in browser After I created a new Play project (Scala), I navigated to http://localhost:9000/@documentation/ScalaTodoList. Every single quote character on the page is rendered as 4 question marks, so I assume there is some funny text encoding going on. Since I am both a Scala and Play newbie, I have really no idea where to start looking. Anyone who has a clue? #### Spark streaming - saveAsTextFile on DStream I am new to Spark and Spark streaming. I am trying to save a DStream to file but can't figure out how to do it with provided methods on DStream (saveAsTextFiles). Following is what I am trying to do Eg. DStream of type DStream[(String, String)] ("file1.txt", "msg_a"), ("file1.txt", "msg_b"), ("file2.txt", "msg_c"), ("file2.txt", "msg_d") I want the messages msg_a & msg_b to be stored in file1.txt and msg_c & msg_d in file2.txt I got as far as grouping the stream on the first item in tuple but I can't figure out how to get the file name out of the stream and pass that to saveAsTextFile() method on DStream Following is what I came up with so far. I am consuming messages from Kafka. object SparkStreamingTest2 { val sparkHome = "/Users/parth/Projects/spark-0.8.0-incubating" def main(args: Array[String]) { if (args.length < 5) { System.err.println("Usage: KafkaWordCount <master> <zkQuorum> <group> <topics> <numThreads>") System.exit(1) } val Array(master, zkQuorum, zkGroup, topics, numThreads) = args val ssc = new StreamingContext(master, "KafkaWordCount", Seconds(4), sparkHome, List("target/scala-2.9.3/sparktest_2.9.3-0.0.1.jar")) ssc.checkpoint("checkpoint") val topicpMap = topics.split(",").map((_,numThreads.toInt)).toMap // lines are in format "filename.txt#msg_x" val lines: DStream[String] = ssc.kafkaStream(zkQuorum, zkGroup, topicpMap) val fileNameAndMsgTuples = lines.map { line => val Array(fileName, message) = line.split("#") (fileName, message) } val groupedByFileName: DStream[(String, Seq[String])] = fileNameAndMsgTuples.groupByKey() // Save message groups to file ?? } }  Thanks for your help in advance! ### CompsciOverflow #### In cache-oblivious algorithms, how is recursive reduction of data performed? From wikipedia: "Typically, a cache-oblivious algorithm works by a recursive divide and conquer algorithm, where the problem is divided into smaller and smaller subproblems. Eventually, one reaches a subproblem size that fits into cache, regardless of the cache size." (Is this recursive subdivision manual, or automated? Assuming automated...) In what way must one produce code in order for it to be suitable for reduction to a cache-oblivious form? Does this rely on the specific implementation of recursive reduction applicable in one's given case? Another way of putting the question is, would I need to write my code in a special way in order for some automated parser to run through the code and optimise it for cache-oblivious access? P.S. I'm trying to get a handle on how to approach writing code for practical optimisation using the methods noted in the literature. ### StackOverflow #### Scala puzzler: Why doesn't head work with toSet whereas it works with Set? Can some explain why the following happens? I mean if something has the type of String, then I expect to run head on it. But Set("ab").head works, whereas List("ab").toSet.head.head doesn't. Why? $ scala
Welcome to Scala version 2.10.3 (Java HotSpot(TM) Server VM, Java 1.7.0_25).
Type in expressions to have them evaluated.

scala> List("ab").toSet
res0: scala.collection.immutable.Set[String] = Set(ab)

scala> Set("ab")
res1: scala.collection.immutable.Set[String] = Set(ab)

res2: String = ab

res3: String = ab

<console>:8: error: value head is not a member of type parameter B
^

res5: Char = a


I'm new to ZMQ trying to figure out how to integrate it in my project. I need to implement the most basic request-reply model where the server is Java and the client is C++. I'm working on Windows using VS2013 and Eclipse (STS). I find the material on the 0MQ overwhelming and confusing. What I'm looking for is a simple list of steps for:

2. What (location and files) do I need to get into my Eclipse project so that I can begin using 0MQ in my code.

4. What (location and files) do I need to get into my VS project so that I can begin using 0MQ in my code.

Thanks.

#### LinearSeqOptimized#find Reference Copy

Looking at the Scala 2.10.0's implementation of LinearSeqOptimized#find in LinearSeqOptimized.scala, why is it necessary to call var these = this?

Why couldn't this simply be used?

  override /*IterableLike*/
def find(p: A => Boolean): Option[A] = {
var these = this
while (!these.isEmpty) {
these = these.tail
}
None
}


### TheoryOverflow

#### 2DFA that requires many states in equivalent DFA?

Is there a 2DFA with $n$ states (where $n$ is nontrivial, say at least 4) that requires at least $2^n$ states to simulate using any DFA?

A two-way DFA (2DFA) is a deterministic finite-state automaton that is allowed to move back and forth on its read-only input tape, unlike finite-state automata that may only move the input head in one direction.

It is well-known that 2DFAs recognize precisely the same class of languages as DFAs, in other words the regular languages. Less well-understood is the question of how efficient the simulation is. The original constructions from the late 1950s by Rabin/Scott and Shepherdson used a notion of crossing sequences and are quite hard to analyse. Moshe Vardi published another construction that shows an upper bound of $2^{O(n^2)}$ states, but this bound may have some slack.

I am asking whether any (families of) 2DFAs are known that require many states in any DFA simulating them, even after Myhill-Nerode minimization of the DFA. Moreover, would there be any interesting consequences of knowing such 2DFAs?

### StackOverflow

#### Scala way to add to List if not null

Scala newbie here...

Doing below on address java object with getStLine*() methods... yields a List with empty strings for the empty strings returned. whats the Scala way to cleanly not add empty strings to a list.

val streets = List[String](addr.getStLine1, addr.getStLine2, addr.getStLine3)


### CompsciOverflow

#### Decision Problem Algorithm

I have a question: Every Decision problem has a method, turing machine or algorithm to solve it? If the answer is not, Could show me any example?

### StackOverflow

#### How do convert this code and my thinking to a functional mindset (Clojure)?

How do I convert this JavaScript code to Clojure?

I am trying to draw a (x,y) world where the cells are on or off according to the fill property. In the example below I am trying to print the rows then columns but my next step is to move the fill property around (up, down, left, right). So, I don't want an answer which wouldn't work if I were not printing the data structure.

My goal is to understand how to think about this problem in a functional way. It was easy for me to solve this problem with mutable state in JavaScript. While working on the solution, I was able to easily reason about how to write the code in JavaScript but when I came to do the same in Clojure, I was at a loss. So, I feel like a good answer to this question would be to help me understand how to think about this kind of problem in a functional way.

I have watched many talks and read a bunch of articles about Clojure and functional programming but when it becomes to writing the code, it's difficult for me to get started.

var world = [
[{ x: 1, y: 4, fill:0 }, { x: 2, y: 4, fill:0 }, { x: 3, y: 4, fill:0 }, { x: 4, y: 4, fill:0 }],
[{ x: 1, y: 3, fill:0 }, { x: 2, y: 3, fill:0 }, { x: 3, y: 3, fill:0 }, { x: 4, y: 3, fill:0 }],
[{ x: 1, y: 2, fill:0 }, { x: 2, y: 2, fill:0 }, { x: 3, y: 2, fill:0 }, { x: 4, y: 2, fill:0 }],
[{ x: 1, y: 1, fill:0 }, { x: 2, y: 1, fill:0 }, { x: 3, y: 1, fill:1 }, { x: 4, y: 1, fill:0 }]
];

function printworld(world) {
var out = "";
for(var row=0; row<world.length; row++) {
for(var col=0; col<world[row].length; col++) {
out += "["+world[row][col].fill+"]";
}
out += "\n"
}
console.log(out);
}

printworld(world);


The output looks like:

[0][0][0][0]
[0][0][0][0]
[0][0][0][0]
[0][0][1][0]


### StackOverflow

#### Integrating Scala into an existing project in Java

we have a project written in Java. It is a maven project, that has jsp pages and a lot of java code:

• servlets for handling user requests and dealing out responses.
• classes of logic used by servlets in order to process the required logic.
• classes of SQL generators used to connect to database and perform the specific queries required by the logic.

Well, we are looking towards Scala. For example, we use mappers in order to get a collection of entities from the database and after that transform this collection, filter it and return to the servlet. That seems like a good thing to do in Java. The question is - how can I integrate that into an existing project? Can I use in my logic files written in Java some Scala classes that again use the mapper files written in Java?

#### Why does Scala not allow a parameter name in a function type definition?

Scala does not allow one to say:

def m(f:(numer:Double,denom:Double)=>tan:Double) = {...}


Just like annotating variables with types means a variable at least has some documentation, so would allowing the variables in a function type definition provide some documentation. Since it would be optional, the programmer would decide when to do it. But the above is definitely more informative than:

def m(f:(Double,Double)=>Double) = {...}


Would this added flexibility break the language syntax?

### QuantOverflow

#### Min VaR and Min TE as second order cone program

The quadratic optimization (min variance) $$w^{T} \Sigma w \rightarrow \text{min},$$ where $w$ is the vector of portfolio weights and $\Sigma$ is the covariance matrix of asset returns, is a well studied problem. In practice we have to define certain constraints. These are easy linear, continuous ones (like $\sum_{i=1}^n w_i = 1$ or constraints on turn over) or difficult binary constraints (if an asset is bought than with at least $x\%$, a cardinality constraint, etc.).

Setting $\hat{w} = w_{Pf} - w_{BM}$ as the vector of active weights of the portfolio relative to a benchmark, the above formula describes minimizing the (squared) tracking error.

The difficult binary constraints lead to mixed integer programming (MIP). Commercial solvers solve this by methods such as branch and bound which are very time consuming for large problems (e.g. 1000+ assets).

I heard of an approach to formulate such problems as second order cone programs (SOCP) and that such problems are usually solved more efficiently.

I have plenty of experience with branche and bound and I would like to switch to SOCP. Do you know of any good refernce of SOCP and portfolio/TE optimization with hard real world constraints (especially with binary variables)? Do you have any experiences whether switching is worth the efforts?

PS: Let us assume that $\Sigma$ is well estimated and that variance makes sense. I know that this is debatable ...

EDIT: This paper Applications of second-order cone programming describes the formulation of a quadratically constrainted quadratic program as SOCP. Will also have a look here.

### StackOverflow

#### Better display of boolean formulas

I want to implement a method for showing a propositional formula in SML. The solutions that I found so far was of this type:

fun show (Atom a) = a
| show (Neg p) = "(~ " ^ show p ^ ")"
| show (Conj(p,q)) = "(" ^ show p ^ " & " ^ show q ^ ")"
| show (Disj(p,q)) = "(" ^ show p ^ " | " ^ show q ^ ")";


This produces unnecessary braces:

((~p) & (q | r))


when, what I'd like to have is:

~ p & (q | r)


I saw, that Haskell has a function (display?) which does this nicely. Can someone help me out a little bit. How should I go about this?

#### Creating a lazy hashed iterator

In research of this question I peeked at the Iterators chapter in the book Higher Order Perl, and some of the material there was a bit over my head and didn't think necessarily addressed what I specifically want here.

What I mean by lazy hashed iterator is a way to create a structure that would emulate this behavior:

%Ds = {
'1' => 1 .. 20;
'2' => 21 .. 40;
'3' => 41 .. 60;
'4' => 61 .. 80;
...
}


Unfortunately, since this is a hash it would not be in order and thus useless in case of very large numbers.

The behavior is this:

I have a number.
I need to compare it with a sequence of ranges and as a result of the comparison the
code/sub would return another number that is the "key" of that range in case the
number is in that range. (>= with the beginning or <= with the end point of said range)
The "key" of the ranges are numbers from 1..2..3 and so on.
The code/sub will always return for a positive integer no matter how large it is.


By implementing this all lazily I mean if there is a way to emulate this behavior and not compute the sequences of ranges with their respective "keys" with every call of the sub or iteration of a loop. Basically compute once.

Yes it's true that I could choose a maximum boundary, hardcode this in a loop and be done with it but the problem is I don't know of how many of these steps I would need in the end.

Is there a way to do this with perl constructs or maybe perhaps there is a CPAN module that offers this kind of behaviour and my simple search of it didn't uncover it.

Here is a piece of code that illustrates what I mean:

sub get_nr {
my $nr = shift; my %ds = map {$a = '1' if /1/ .. /20/;
$a = '2' if /21/ .. /40/;$a = '3' if /41/ .. /60/;
$a = '4' if /61/ .. /80/;$_ => $a } 1 .. 80; while (my ($k, $v) = each %ds) { if ($k == $nr){ print "number is in range$v \n";
}
}
}


The output for:

 get_nr(4);
get_nr(15);
get_nr(22);
get_nr(45);


Is:

number is in range 1
number is in range 1
number is in range 2
number is in range 3


### QuantOverflow

#### Help with understanding a normal distribution/probability question

http://www.ntuzov.com/Nik_Site/Niks_files/Research/papers/stat_arb/Ahmed_2009.pdf

Specifically:

When the order rates are time-varying probabilities must be computed via Monte Carlo. A simple algorithm is as follows. There are six types of orders; buy/sell and market/limit/cancel. For each type of order there are multiple rates depending on the distance to the bid/ask, i.e. if the bid is at the tenth highest tick level then there are ten limit buy orders.

Let $\lambda_i$, $i \in \mathcal{I}$ be the collection of all order rates and $\boldsymbol{x}_t = (x_1, \ldots, x_n)$ be the current state of the order book, as specified in [1]. Then there are a fixed and finite number of possible states $x_t+1$ can take on. The next state of the order book is completely determined by which order arrives first. It is known that if $X_i \sim \exp(\lambda_i)$ then

Therefore to determine the next state of the order book we just sample $u \sim U(0,1)$ then partition the interval $(0, 1)$ according to the above probabilities to determine which order arrived first. After the next state of the order book is computed we recompute the $\lambda_i$'s since they depend on the order book, i.e. $x_t$ is an inhomogeneous Markov chain, and repeat to generate an entire sample path.

Let $A$ be the set of $\omega$ where the midprice increases, to compute its probability we simulate sample paths until there is a change in the midprice and compute $I_A(\omega)$ then estimate the probability as

EDIT:

Ok, I have made some progress:

As the comment below says, X is exponentially distributed. However, I do not get what calculating the "distribution of the minimum exponential random variable" is for?

Also, once we have done this we then (seem to) plot the uniform distribution between 0 and 1 and then plot the probabilities on the x-axis, and, I think, look for the probability with the greatest area?

I really don't understand why this tells us the next state?? What exactly is finding the minimum exponential random variable telling us?

Why do we need to use the uniform distribution?

### CompsciOverflow

#### Partition an infinite regular language into 2 disjoint infinite regular languages

Given any infinite regular language $L$, how can I prove that $L$ can be partitioned into 2 disjoint infinite regular languages $L_1, L_2$? That is: $L_1 \cup L_2 = L$, $L_1 \cap L_2 = \varnothing$, and $L_1$ and $L_2$ are both both infinite and regular.

So far, I thought of:

1. using the pumping lemma such that $$\begin{gather} L_1 &= \{ xy^nz \mid \text{$$n$$ is even} \} \\ L_2 &= \{ xy^mz \mid \text{$$m$$ is odd} \} \\ \end{gather}$$ but couldn't prove that they are dijoint or covering $L$ completely.

2. Using the regular language partitions $\Sigma^*$ into dijoint equivalence classes, but I haven't figured out how to determine if an equivalence class is regular or infinite.

### Fefe

#### Die Privatsender wollen an den Kuchen mit der "Vergütung ...

Die Privatsender wollen an den Kuchen mit der "Vergütung für Privatkopien" ran. Ich vermute mal, dass die Gebühren auf Medien hochgehen werden.

#### Ausrede des Tages: Schuld an den Pro-EU-Demonstrationen ...

Ausrede des Tages: Schuld an den Pro-EU-Demonstrationen in der Ukraine ist das kalte Wetter. Denn, äh, das weiß doch jeder, dass bei kaltem Wetter die Leute bevorzugt auf die Straße gehen und sich stundenlang den Arsch abfrieren!

Die Ausrede kommt übrigens vom russischen Staatsfernsehen.

### Planet Theory

#### Faculty positions at Harvard

Tenured faculty position in Applied Math

Tenure-track faculty position in Computer Science

#### Postdocs at Harvard

Postdocs & Visiting Scholar positions in Center for Research on Computation & Society (includes opportunities for TCS folks interested in privacy or econ)
Deadline: December 13, 2013 (VERY SOON!)

http://crcs.seas.harvard.edu/apply

Postdocs in Theory of Computation group

http://toc.seas.harvard.edu/call-for-postdocs

### StackOverflow

#### Haskell for Lambda Calculus, Type Inferencing

My adventure in Haskell programming hasn't been all epic. I am implementing Simple Lambda Calculus, and I am glad to have finished Syntax, Evaluation, as well as Substitution, hoping they are correct. What remains is typing as defined inside the red box (in the figure below), for which I am looking for guidance.

Please correct me if I am wrong,

(1) but what I gathered is that (T-Var), returns the type of given a variable x. What construct in Haskell returns type ? I know that in prelude it is :t x, but I am looking for one that works under main = do.

(2) If I were to define a function type_of, it's most likely that I need to define the expected and return type, as an example, type_of (Var x) :: type1 -> type2

type1 should be generic and type2 must be whatever object type that stores type information of a variable. For this, I am lost on how to define type1 and type2.

(3) For (T-APP) and (T-ABS), I assume I apply substitution on Abstraction String Lambda and Application Lambda Lambda respectively. The type of the reduced form is the returned type. Is that correct?

### CompsciOverflow

#### Eliminating unit productions from CFG

I read about eliminating unit productions in my book but I had a doubt with the following CFG -

S->A|11
A->B|1
B->S|0


Now algorithm says to remove all the unit productions and copy the productions so here is what i get

S->11|1|0


and rest of them are useless because A & B are not reachable from .

Now the doubt is does this accepts exactly same language? coz with previous one I can have 111111 accepted but I can't in this transformed one.

Please correct me if I am wrong

Thanks

### /r/compsci

#### Curious to know what other coders/programmers think...

So, I'm studying comp. eng. at my school. I'm more interested in hardware than software, but I still enjoy coding from time to time. I had a programming class this semester that had assignments and multiple choice only exams as the sole evaluation.

I was wondering what people think of multiple choice exams as a means to evaluate a person's coding capabilities? I just wrote an exam with some extremely convoluted questions and ambiguity which don't seem like a fair assessment of anyone.

I find that they're not truly indicative of what a student is capable of because, for me at least, programming is about creativity and resourcefulness. When writing a program, you have access to references and to an IDE that lets you test code and come up with the best code for your desired outcome. I don't think that using the resources available as opposed to know everything off the top of your head means you're a lesser programmer.

submitted by LordNagafen

### /r/emacs

#### Emacs 24.3 Causing "Unknown Publisher" Error in Windows 7

I've just installed Emacs 24.3 by downloading the zip fill and unzipping it into a folder in C:\Program Files (x86). Now when I execute anything in the bin folder, I get an "Unknown Publisher" warning that I have to click through. I didn't (don't) have this problem with v23, which is installed into a parallel folder. Any ideas on what I need to do? Googling around suggests that there isn't an easy way to disable this on a program-by-program basis, and I'm loathe to turn off checking for all .exe files.

submitted by srt19170

### StackOverflow

#### Which CI server works with SBT?

I'm considering using SBT for a new scala project, but I'm not sure which CI server - hudson / cruise / whatever has support for it. Any ideas?

I know SBT is a little Maven like, but I don't think it can work as a drop in replacement for Maven in the eyes of a CI server.

#### Correct idiom for a conditional return in scala

I am trying to figure out the scala way to implment something I would do all the time in java. In java I would have snarf_image (below) return null if it meets the if condition, otherwise return the bArray. What is the scala way to do this? This code doesnt even compile and I cant figure out the right way to do it - Im sure my thinking is off.

 def snarf_image ( sUrl : String ) : Array[Byte] = {
val bis = new BufferedInputStream(new URL(sUrl.replace(" ", "%20")).openStream())
val img = ImageProcessing.ArrayToImage(bArray)
if ( img.getHeight < 100 || img.getWidth < 100 ) {
Empty
} else {
bArray
}
}


For the record I am using lift (hence using empty) but Im pretty sure this is more a scala question.

### CompsciOverflow

#### does a DFA converted from NFA always contain 2^Q states?

when converting an NFA to DFA, we create sub-sets of states in the NFA. does it mean that every DFA-converted-from-NFA contain 2^Q states? or if some sub-sets are unreachable then they are not included in it?

### StackOverflow

#### Is is possible to make SBT task use a specific configuration? How to run SBT commands in tasks?

def myTask = Task <<= (Task1, Task2, Task3) map {(_,_,_)=>;}


Task1,Task2,Task3 as an arguments taking config variable tcWebApp which is a directory.

 tcWebApp := file("../tomcat")


Everything works fine, but what I need now is to create one more task myTask2, similar to myTask but I would like to invoke this task with other direction in tcWebApp. Is it even possible ? I've tried something like

tcWebApp in myTask2 := file("newDir")


but it's not working.

Second idea was to execute SBT command "set" in task:

 set tcWebApp := file("newDir")


but i don't know how to execute SBT commands from task and I'm not pretty sure is it possible.

### StackOverflow

#### Streaming data to/from Play framework on an open connection

I need to send a stream of data to Play server. The length of the stream is unknown and I need to get a response every line break \n or for every several lines. Rather then wait for the whole data to be sent.

Think of the following usecase:
lets say i'm intended to write a console application, that when launched, connects to my web server, and all the user input are being sent to play on every line break, and gets responded asynchronously. All above should be performed on a single connection, i.e. I don't want to open a new connection on every request I send to Play (a good analog would be 2 processes communicating through 2 pipes).
What is the best way to achieve this?
And is it possible to achieve with a client that communicates with the server only via http (with a single http connection)?

EDIT:
my current thoughts on how to approach this are as follows:
i can define a new BodyParser[Future[String]] which is basically an Iteratee[Array[Byte],Future[String]]. while the parsing takes place, i can compute the result asynchronously and the action can return the result as ChunkedResult in the future's onComplete method.

does this sound like the right approach?
any suggestions on how to achieve this?

#### scala code coverage using scct, how to exclude files by regex

using scct in my project there are some files I would like to be excluded

my build file :

object myBuild extends Build {

lazy val americanNinja1 = Project(id = "American-Ninja", base = file("americanNinja")).settings(ScctPlugin.instrumentSettings: _*)

//more code .....
}


how can i add setting to the scct to exclude all .html files (for example)

looking at the github project of scct i see that there is a filter option

https://github.com/sqality/scct/blob/master/src/main/scala/com/sqality/scct/CoverageFilter.scala

and there was an issue that was solved for this matter :

https://github.com/mtkopone/scct/issues/60

any idea how to do this ?

Thanks!

#### Which version of Java does SBT use?

How to find out, which version of JDK SBT uses?

My laptop has both JDK 1.6 and JDK 1.7 installed, so I was just wondering.

#### exclude jars with SBT assemble plugin

I am struggling to give exclude pattern in build.scala. I have seen few posts where they mentioned about build.sbt. I am new to sbt. Can someone help me to write exclude pattern in build.scala.

I want to exclude below two packages which causes error while running assembly : xmlbeans and xml-apis

Error :
[trace] Stack trace suppressed: run last app/*:assembly for the full output.
[error] (app/*:assembly) deduplicate: different file contents found in the following:
[error] /Users/rajeevprasanna/.ivy2/cache/org.apache.xmlbeans/xmlbeans/jars/xmlbeans-2.3.0.jar:org/w3c/dom/TypeInfo.class
[error] /Users/rajeevprasanna/.ivy2/cache/xml-apis/xml-apis/jars/xml-apis-1.3.03.jar:org/w3c/dom/TypeInfo.class


I wrote build.scala in reference to this file : https://github.com/eed3si9n/sbt-assembly-full-config-sample/blob/master/project/builds.scala

### CompsciOverflow

#### Fingerprint Artifical Neural Network

We have a project to develop an AFIS(Automated Fingerprint..) We have read many papers on the field and still reading Handbook of Fingerprint Recognition. What we arent able to understand is ; Where are we exactly going to use Neural Networks?

So far we get ; 1)Apply Histogram Eq. + DFT(if necessary) to get contrast and remove noise. 2)Image Binarization + 3)Image Thinning (Morphological Thinning) These are okay. After that , there is feature extraction. Working in 3x3 windows we search for minutiae points , looking for patterns like - 1 in middle , and only has one other 1 neighbor so it is termination- . Then we get minutiae points and we use Poincare Index method to get singular points. But after that , while we have minutiae's and singulars , where are we going to use neural network? If for classification , how? Since we extracted singular points with poincare index why do we need ANN to classify? If for extracting minutiae points , haven't we done it with feature extraction? Any resources you might want to point out? Thanks.

### StackOverflow

#### Does with clause in Korma fire one query per item in relationship?

The docs at Korma page says, under entities,

(has-many email)
;; assumes users.id = email.users_id
;; but gets the results in a second query
;; for each element


So, if I am understanding it correctly if one teacher has many students and I want all the students for a teacher nested within each teacher map, Korma will generate one SQL query per teacher ?

Won't this hurt the performance when we have like millions of teachers ?

#### could not find implicit value for parameter flash: play.api.mvc.Flash

I have based the app on a couple of the sample apps but im getting the error: could not find implicit value for parameter flash: play.api.mvc.Flash

(i want the flash to satify the templates @(title: String, user: User, systems: List[System], currentOrderBy: Int, currentFilter: String)(implicit flash: play.api.mvc.Flash) signature so I can pass the prior results in)

this is my code that give the issue:

def index = IsAuthenticated { username => _ =>
Ok(
html.systems.systems("Ken2::Systems",
user,
System.all,
1, "%"
)
)
}.getOrElse(Forbidden)
}


in the documentationn it says : If the error ‘could not find implicit value for parameter flash: play.api.mvc.Flash’ is raised then this is because your Action didn’t import a request object. Add an “implicit request=>” as show below:

def index() = Action {
implicit request =>
Ok(views.html.Application.index())
}


but where in my code would i stick this implicit request ? (the format of my code comes from one of the sample apps)

thank you

#### What are the differences between mapcat in Clojure and flatMap in Scala in terms of what they operate on?

I understand the equivalent to flatMap in Scala is mapcat in Clojure.

I have an inkling that mapcat in clojure only works with sequences, unlike flatMap in Scala which is more flexible.

My question is - what are the differences between mapcat in Clojure and flatMap in Scala in terms of what they operate on?

Assumptions:

• I understand that Scala has a rich type system and Clojure has optional typing - I'm interested to know if the is a limitation in the parameters that mapcat accepts that make it only have a subset of flatMaps functionality.

#### Lisp in the real world

I have experimented with Lisp (actually Scheme) and found it to be a very beautiful language that I am interested in learning more about. However, it appears that Lisp is never used in serious projects, and I haven't seen it listed as a desired skill on any job posting. I am interested in hearing from anyone who has used Lisp or seen it used in the "real world", or who knows whether it is considered a purely academic language.

### QuantOverflow

#### Rate of Return Required on Buying Stocks with Loan

A bank gives a loan $L$ for $m$ months and the monthly interest rate is $i$.
The bank requires monthly installments - which I calculate is $I = \frac{L}{m} + Li$.

I use this loan to buy stocks. If the stock price is $p(m = 0)$, then I can buy $\frac{L}{p(m = 0)}$ shares. I plan to pay the $m$ monthly installments only by selling the optimal number of stocks every $x$ months.

How can I calculate if this plan will pay off? How do I optimize $x$? What variables should I be looking at? Can profit be made from this example? How much does the stock price have to rise to break even? I match abbreviations with their referents.

If I just accept the loan of $L$ and do nothing with it except return it in $m$ installments, then my loss is only the total interest $=m \times Li$. So to break even, $m \times Li$ is what I need to gain from my stocks. But this feels too simple and naive?

Furthermore, the rate of return over the entire period = total interest/loan amount $= \frac{mi}{L}$? This feels wrong because it doesn't account for the fact that stocks will be sold every $x$ months to cover the $m$ monthly installments? I think $x\frac{ I}{p(m)}$ shares must be sold to cover $x$ monthly installments?

I'm already getting lost so maybe some numbers will help - suppose
☻ $L = 10,000$ USD,
☻ $m = 24$,
☻ $i = 0.25\%$
☻ $p(m = 0) = 80$,
☻ The stock price is expected to go up to $100$ within the 24 months.

I'm new to financial math and only know first-year undergrad math. Sorry for any faults.

### StackOverflow

I would like to read the book Coding the Matrix. They use Python in the book and I am more at ease with Haskell. Is it a good idea to read the book anyway using Haskell?

### StackOverflow

#### isInstanceOf for multiple possibilities

I am able to check whether an object o is an instance of A or B like that:

o.isInstanceOf[A] || o.isInstanceOf[B]


But this gets ugly if i have more possiblities.

Is there a way to define something like this:

o.isInstanceOf[A,B,C,D, ...]


or

o.isInstanceOf(A,B,C,D, ...)


or

o.isInstanceOf(Set(A,B,C,D, ...))


### StackOverflow

#### Scala: Using supertype operator in pattern matching

I have made my own framework of traits and classes that extend my traits. The parent of all classes is a trait named 'Contract'. 'Combinator' and 'ElementaryContract' are two immediate children of Contract.

def disintegrateContract[T](element: T): Elem =
{

element match
{
case com <: Combinator =>  matchCombinator(com)
case e <:ElementaryContract =>matchElementaryContract(e)
}
}


I want to make a match class that recognizes whether a passed 'Contract' is a subtype of 'Combinator' or 'ElementaryContract' and then pass it to some other functions. This is the compilation error I get:

 '=>' expected but '<:' found


Probably it does not recognize the subtype operator. How can I make this happen?

### CompsciOverflow

#### Intersection of context free with regular languages

The intersection of a context free language L with a regular language M, is said to be always context free. I understood the cross product construction proof, but I still don't get why it is context free but not regular.

The language generated by such an intersection has strings that are accepted both by a PDA and a DFA. Since it is accepted by a DFA, shouldn't it be a regular language? Plus, if the intersection is regular, it also implies context free, since all regular languages are also context free.

Can someone explain to me why the language obtained by such an intersection is not regular?

Thanks.

### StackOverflow

#### Am I still fit for play without Scala [on hold]

I'm following on from this question: Can we completely avoid Scala and simply use the Play! framework which discusses the possibility of using Play with out diving into Scala.

I was hoping to get some advice on not only what is possible but what is recommended. From the question linked, I gather that in Play V2 it is perfectly simple to use Play without knowing Scala. At first I thought - great as for now I still do not know Scala and would like to get started with Play ASAP. However,

1. Is this a good idea? Will avoiding Scala end up just trying to convert a framework into something it isn't? Or does it not matter as there are other benefits of Play without Scala?

2. What about JSP - is it supported natively? Can I render it easily - cause I already know the basics of it?

3. For long term use, should I learn Scala if I want to get into web programming? Or will java/JSP be enough for java like frameworks?

Any advice on what is supported and how one should best make use of it will be greatly appreciated.

Sorry just to add: The process of taking in a form, extracting parameters and returning a response - how much Scala is used? Used for processing or only for rendering? And If I render using Scala does that nullify the need for JSP?

### UnixOverflow

#### How to secure Apache/MySql/PHPMyAdmin against brute force/dictionary attacks? (Fail2ban?)

I'm wanting to run a publicly accessible server for web hosting. I'm interested in preventing brute force attacks made against PHPMyAdmin or any other form of authentication process I may include in the site.

What types of options exist for this without having to monitor every possible login method that may exist that I actually want public facing?

I use Fail2ban to block things like too many ssh attempts. I also just started using it for stopping too many PHPMyAdmin login attempts. My concern are other arbitrary login pages that my web developer may create for things specific to the website.

Initial thoughts were things like stopping too many requests per X seconds. That, however, would have to be a rather high number to accommodate pages with lots of images and sounds like it would have a lot of room for false positives.

I would think this would be similar to how a shared hosting provider would help keep your website secure when they have no idea what content you may have (a bank would obviously not host that kind of information with a shared provider, but many less sensitive sites would still want to keep logins from being brute forced). Perhaps this is the reason for captchas?

### TheoryOverflow

#### 3-sat to 2-sat reduction [on hold]

It is known that 3-SAT belong to - NP-Complete complexity problems, while 2-SAT belong to P as there is known polynomial solution to it.

So you can state that there is no such reduction from 3-SAT to 2-SAT unless $P=NP$.

I am looking for strong proof for this state, regardless NP belong to P or not.

### QuantOverflow

#### When did volatilities start to smile in capital markets?

Glimpsing through literature, I read that volatilities in the equity market started to display a smile after the crash in 1987. But when did volatilities start to smile in capital markets?

### StackOverflow

#### Is application level heartbeating preferable to TCP keepalives?

Is there a reason why I should use application level heartbeating instead of TCP keepalives to detect stale connections, given that only Windows and Linux machines are involved in our setup?

#### Adding Strings to an Array out of a List in a Scala template

I have a List of Objects and want to iterate every Element in this List to get their Id Strings. Those Strings must be saved into another List. I always get this Compiler Error:

    [error] FilePath:line:64: illegal start of simple expression
[error]                 @var formNameList : Array[String] = new Array[String](formList.size())
[error]                  ^
[error] File Path:69: ')' expected but '}' found.
[error]     }
[error]       ^
[error] two errors found
[error] (compile:compile) Compilation failed
[error] Total time: 3 s, completed 05.12.2013 14:03:37


So please guys help, before I drive insane.

My Code:

@var formNameList : Array[String] = new Array[String](formList.size())
@for(i <- 0 until formList.size()) {
}
@views.html.formmanager.showresults(formNameList, formManager)


Im a Newbie in Scala and this is a very simple Task in Java but Scala is such a tough language. Its also very hard to read: What does this .:::, ::: or this <++= mean?

#### Generating function that will generate stream of mailboxes

I am still in the processing of learning SML and Concunrrent ML, i want to know how to create new channels recursively.

I need to create a generating function that will generate a stream of mailboxes. The generating function will be parametrized by two channels, the in and out channels for the first mailbox, and integer dictating how many mailboxes to generate. This function should be recursive and allocate new channels. On the recursive invocation of generator the previous out channel should be the in channel for the next mailbox. A new channel should be allocated as the new out channel.

Can someone throw some light on this please?

I want an understanding on the creation of the new channels

### TheoryOverflow

#### Proof verification: $P \neq NP$ [on hold]

I think that I can prove that $P \neq NP$, please verify my proof.

1. Any NP-Complete problem could be reduced to Circuit SAT. Any NP-Complete problem have a deterministically polynomial time verifiable solution or in other words, we can "quickly" verify the solution to the optimization version of the problem, it can be expressed as a Satisfiability problem which asks for: what input satisfies the verification conditions - this is a satisfiability problem, what remains is to "booleanize" it, as one would when making a logic circuit to verify the solution.(1)
2. Any circuit sat could be reduced to 3-SAT
3. Any problem with solution time of $O(N^C)$ where $C$ is a neutral number, could be reduced to $O(N^{C-1})$ in polynomial time
4. Any problem with linear solution time, could be reduced to 2-SAT.
5. Any problem in $P$ could be reduced to 2-SAT.
6. Lets assume that $P=NP$ and any problem in $NP$ could be reduced to problem in $P$
7. 3-SAT can be reduced to 2-SAT. And solved this is a contradiction to (6).

## Notes

• 3) When I am saying that reduction could be in polynomial time, it does not mean the the reduction will be in linear time, just polynomial.
• 7) Even the most simple instance of 3-sat, $(a \lor b \lor c)$ could not be represented in any form of 2-sat. There is strong dependency between all the 3 variables that will be lost when you will try create alternative dependencies with 2 variables. More than that, if that was possible then $NP$ would indeed belong to $P$. In my proof I show that it is if and only if.

References: 1: Realz Slaw answer to my question: NP-complete: One proof to rule them all

Please provide sincetificant explanation if you found a hole in this proof.

### Planet Theory

#### Bitcoins Revisited

Two years ago I gave a lecture and posted about bitcoin. Of course what I didn't do was buy a bitcoin whose value back then was about $3 and today runs in the$1000 range.

Bitcoins have received quite a bit of press, particularly with the FBI shutting down Silk Road, the drug trafficking site which used bitcoins for their transactions. Then people realized that bitcoins are starting to become a real currency, with a market cap of about US$11 billion not far now from the money supply of the Costa Rica Colones (US$13 billion). Now governments are deciding on how to deal with bitcoins as a currency, one which they really can't regulate or control.

The Economist has one of the better articles on Bitcoins, talking about some of the technical issues involved.
The Bitcoin system is designed to cope with the fact that improvements in computer hardware make it cheaper and faster to perform the mathematical operations, known as hashes, involved in mining. Every 2,016 blocks, or roughly every two weeks, the system calculates how long it would take for blocks to be created at precisely 10-minute intervals, and resets a difficulty factor in the calculation accordingly. As equipment gets faster, in short, mining gets harder. But faster equipment is constantly coming online, reducing the potential rewards for other miners unless they, too, buy more kit. Miners have formed groups that pool processing power and parcel out the ensuing rewards. Once done with ordinary computers, mining shifted to graphics-processing units, which can perform some calculations more efficiently. Miners then moved on to flexible chips that can be configured for particular tasks, called field-programmable gate arrays. In the past year, bespoke chips called ASICs (application-specific integrated circuits) have appeared on the scene.
Then there was the paper Dorit Ron and Adi Shamir wrote that explored the bitcoin transaction graph and suggested a (now supposedly debunked) connection between the mysterious creator of bitcoins, "Santoshi Nakamoto", and Ross William Ulbricht aka Dread Pirate Roberts, the founder of Silk Road.

Bitcoins even make it to my Thanksgiving table. My brother thought they were a scam even though I pointed out the systems has no scammers. He remains unconvinced though he invests heavily in gold, which has the same property of the value mostly being there because people believe it has value.

I taught a lecture on bitcoins again in my intro theory course. We all generally agreed that a few years from now we'll all remember the days when bitcoins were worth $1000. Not sure we'll remember those days because bitcoins will be worth millions or because they'll be worth pennies. ### TheoryOverflow #### How hard is unshuffling a string? A shuffle of two strings is formed by interspersing the characters into a new string, keeping the characters of each string in order. For example, MISSISSIPPI is a shuffle of MISIPP and SSISI. Let me call a string square if it is a shuffle of two identical strings. For example, ABCABDCD is square, because it is a shuffle of ABCD and ABCD, but the string ABCDDCBA is not square. Is there a fast algorithm to determine whether a string is square, or is it NP-hard? The obvious dynamic programming approach doesn't seem to work. Even the following special cases appear to be hard: (1) strings in which each character appears at most four six times, and (2) strings with only two distinct characters. As Per Austrin points out below, the special case where each character occurs at most four times can be reduced to 2SAT. Update: This problem has another formulation that may make a hardness proof easier. Consider a graph G whose vertices are the integers 1 through n; identify each edge with the real interval between its endpoints. We say that two edges of G are nested if one interval properly contains the other. For example, the edges (1,5) and (2,3) are nested, but (1,3) and (5,6) are not, and (1,5) and (2,8) are not. A matching in G is non-nested if no pair of edges is nested. Is there a fast algorithm to determine whether G has a non-nested perfect matching, or is that problem NP-hard? • Unshuffling a string is equivalent to finding a non-nested perfect matching in a disjoint union of cliques (with edges between equal characters). In particular, unshuffling a binary string is equivalent to finding a non-nested perfect matching in a disjoint union of two cliques. But I don't even know if this problem is hard for general graphs, or easy for any interesting classes of graphs. • There is an easy polynomial-time algorithm to find perfect non-crossing matchings. Update (Jun 24, 2013): The problem is solved! There are now two independent proofs that identifying square strings is NP-complete. There is also a simpler proof that finding non-nested perfect matchings is NP_hard, due to Shuai Cheng Li and Ming Li in 2009. See "On two open problems of 2-interval patterns", Theoretical Computer Science 410(24–25):2410–2423, 2009. ### StackOverflow #### Changing date format in syslog Is there anyway we can change the date format in a particular log file being logged to by syslog? I don't want to change the way all logs are being logged, but just by log file. EDIT: I'm using syslogd (in FreeBSD) This is how my file looks like now: Dec 5 07:52:10 Log data 1 Dec 5 07:52:10 Log data 2 Dec 5 07:52:10 Log data 3  This is how I want it to look like: 20131205 07:52:10 Log data 1 20131205 07:52:10 Log data 2 20131205 07:52:10 Log data 3  My syslog.conf looks like this, where /var/log/my_log.log is my logfile: +@ *.notice;local0.none;local1.none;local2.none;authpriv.none;kern.debug;mail.crit;news.err /var/log/messages security.* /var/log/security auth.info;authpriv.info /var/log/auth.log mail.info /var/log/maillog ftp.info /var/log/xferlog cron.* /var/log/cron *.=debug /var/log/debug.log console.info /var/log/console.log local1.info /var/log/my_log.log  ### Lamer News #### How to deploy Go with Traffic on Heroku ### StackOverflow #### How to achieve reliable peer to peer messaging with zeroMQ or some other library out there? By reliable I mean • when subscribers are disconnected we need to be able to send them offline messages. • there could be any number of subscribers • Message queue should be persistent in publisher. So, to deal with that we should have acks from subscribers. We have to have persistent message queue for each subscriber (with what we can do this easy? I heard there is some extensions in postgresql) The main difficulty for me is how to send messages. Should we use sliding window protocol for that? That's a lot of work IMHO, so I'm searching some ready solution for that. It couldn't be, that no one is tried to do that before. ### /r/netsec #### The most comprehensive article on pdf analysis I have seen in a while, plus a tool release! ### StackOverflow #### how to implement validation in case class used as form (using Json.fromJson) i am using TypeSafe activator template as a basis for an app (play+reactivemongo+knockoutJS). i am trying to create a user, and while doing it i want to validate that the country they provided is already in the DB. but due to the structure of the program, i am finding it hard to put in that validation. code Object UserContoller extends Controller { case class UserForm( firstName: String, midName: Option[String], lastName: String, addr1: String, addr2: Option[String], addr3: Option[String], city: String, state: Option[String], zip: String, country: String, email: String, phone: String ) { def toUser: User = User( BSONObjectID.generate, Name(firstName, midName, lastName), Vector(Email(email)), Vector(Phone(phone)), Address(addr1, addr2, addr3, city, state, zip, CountryDao.find(country)) ) } implicit val userFormFormat = Json.format[UserForm] def save = Action(parse.json) { req => Json.fromJson[UserForm](req.body).fold( invalid => { BadRequest("Bad form").flashing(Flash(Map("errors" -> JsError(invalid).toString))) }, form => Async { UserDao.save(form.toUser).map(_ => Created.flashing(Flash(Map("result" -> "success")))) } ) } }  my problem is to-fold: i need to verify the country exists in the DB, and i need to get the country for creating the User (which is an async action). my best idea is to have UserForm implement something that provides a 'fold' method (as in Json...fold), so i can return invalid if the country is not found. if i know it exists, it's probably easier building the MongoDB query, awaiting it and doing a .get on the Option, as i already know it exists. hope i made myself clear. any ideas? ### /r/netsec #### New release for Automater.py (IP/URL/MD5 OSINT Analysis tool) ### QuantOverflow #### Estimate of fx trading volume I understand that there are no official figures of fx trading volume per currency, but are there any good estimates? Is there any fx exchange that is particularly forthcoming with its volume data, say on a monthly basis? Googling the answer has only been partially helpful. (I am not a professional quant, but this is related to professional consulting for a large fund, so I figured this was a good place to ask.) ### StackOverflow #### Custom Json Writes with combinators - not all the fields of the case class are needed I'm trying to write a custom Json serializer in play for a case class but I don't want it to serialize all the fields of the class. I'm pretty new to Scala, so that is surely the problem but this is what I tried so far: case class Foo(a: String, b: Int, c: Double)  Now the default way of doing this, as far as I saw in the examples is: implicit val fooWrites: Writes[Foo] = ( (__ \ "a").write[String] and (__ \ "b").write[Int] (__ \ "c").write[Double] ) (unlift(Foo.unapply))  But what if I want to omit "c" from the Json output? I've tried this so far but it doesn't compile: implicit val fooWritesAlt: Writes[Foo] = ( (__ \ "a").write[String] and (__ \ "b").write[Int] ) (unlift({(f: Foo) => Some((f.a, f.b))}))  Any help is greatly appreciated! ### infra-talk #### CLI testing with RSpec and Cucumber-less Aruba At Bulletproof, we are increasingly finding home brew systems tools are critical to delivering services to customers. These tools are generally wrapping a collection of libraries and other general Open Source tools to solve specific business problems, like automating a service delivery pipeline. Traditionally these systems tools tend to lack good tests (or simply any tests) for a number of reasons: • The tools are quick and dirty • The tools model business processes that are often in flux • The tools are written by systems administrators Sysadmins don't necessarily have a strong background in software development. They are likely proficient in Bash, and have hacked a little Python or Ruby. If they've really gotten into the infrastructure as code thing they might have delved into the innards of Chef and Puppet and been exposed to those projects respective testing frameworks. In a lot of cases, testing is seen as "something I'll get to when I become a real developer". The success of technical businesses can be tied to the quality of their tools. Ask any software developer how they've felt inheriting an untested or undocumented code base, and you'll likely hear wails of horror. Working with such a code base is a painful exercise in frustration. And this is what many sysadmins are doing on a daily basis when hacking on their janky scripts that have evolved to send and read email. So lets build better systems tools: • We want to ensure our systems tools are of a consistent high quality • We want to ensure new functionality doesn't break old functionality • We want to verify we don't introduce regressions • We want to streamline peer review of changes We can achieve much of this by skilling up sysadmins on how to write tests, adopting a developer mindset to write system tools, and provide them a good framework that helps frame questions that can be answered with tests. We want our engineers to feel confident their changes are going to work, and they are consistently meeting our quality standards. ## But what do you test? We've committed to testing, but what exactly do we test? Unit and integration tests are likely not relevant unless the cli tool is large and unwieldy. The user of the tool doesn't care whether the tool is tested. The user cares whether they can achieve a goal. Therefore, the tests should verify that the user can achieve those goals. Acceptance tests are a good fit because we want to treat the cli tool as a black box and test what the user sees. Furthermore, we don't care how the tool is actually built. We can write a generic set of high level tests that are decoupled from the language the tool is implemented in, and refactor the tool to a more appropriate language once we're more familiar with the problem domain. ## How do you test command line applications? Aruba is a great extension to Cucumber that helps you write high level acceptance tests for command line applications, regardless of the language those cli apps are written in. There are actually two parts to Aruba: 1. Pre-defined Cucumber steps for running + verifying behaviour of command line applications locally 2. An API to perform the actual testing, that is called by the Cucumber steps Scenario: create a file Given a file named "foo/bar/example.txt" with: """ hello world """ When I run cat foo/bar/example.txt Then the output should contain exactly "hello world"  The other player in the command line application testing game is serverspec. It can do very similar things to Aruba, and provides some fancy RSpec matchers and helper methods to make the tests look neat and elegant: describe package('httpd') do it { should be_installed } end describe service('httpd') do it { should be_enabled } it { should be_running } end describe port(80) do it { should be_listening } end  The cool thing about serverspec that sets it apart from Aruba is it can test things locally and remotely via SSH. This is useful when testing automation that creates servers somewhere: run the tool, connect to the server created, verify conditions are met. But what happens when we want to test the behaviour of tools that create things both locally and remotely? For local testing Aruba is awesome. For remote testing, serverspec is a great fit. But Aruba is Cucumber, and serverspec is RSpec. Does this mean we have to write and maintain two separate test suites? Given we're trying to encourage people who have traditionally never written tests before to write tests, we want to remove extraneous tooling to make testing as simple as possible. A single test suite is a good start. This test suite should be able to run both local + remote tests, letting us use the powerful built-in tests from Aruba, and the great remote tests from serverspec. There are two obvious ways to slice this: 1. Use serverspec like Aruba - build common steps around serverspec matchers 2. Use the Aruba API without the Cucumber steps We opted for the second approach - use the Aruba API from within RSpec, sans the Cucumber steps. Opinions on Cucumber within Bulletproof R&D are split between love and loathing. There's a reasonable argument to be made that Cucumber adds a layer of abstraction to tests that increases maintenance of tests and slows down development. On the other hand, Cucumber is great for capturing high level user requirements in a format those users are able to understand. Again, given we are trying to keep things as simple as possible, eliminating Cucumber from the testing setup to focus purely on RSpec seemed like a reasonable approach. The path was pretty clear: 1. Do a small amount of grunt work to allow the Aruba API to be used in RSpec 2. Provide small amount of coaching to developers on workflow 3. Let the engineers run wild ## How do you make Aruba work without Cucumber? It turns out this was easier than expected. First you add Aruba to your Gemfile # Gemfile source 'https://rubygems.org' group :development do gem 'rake' gem 'rspec' gem 'aruba' end  Run the obligatory bundle to ensure all dependencies are installed locally: bundle  Add a default Rake task to execute tests, to speed up the developer's workflow, and make tests easy to run from CI: # Rakefile require 'rspec/core/rake_task' RSpec::Core::RakeTask.new(:spec) task :default => [:spec]  Bootstrap the project with RSpec: $ rspec --init


Require and include the Aruba API bits in the specs:

# spec/template_spec.rb

require 'aruba'
require 'aruba/api'

include Aruba::Api


This pulls in just the API helper methods in the Aruba::Api namespace. These are what we'll be using to run commands, test outputs, and inspect files. The include Aruba::Api makes those methods available in the current namespace.

Then we set up PATH so the tests know where executables are:

# spec/template_spec.rb
require 'pathname'

root = Pathname.new(__FILE__).parent.parent

# Allows us to run commands directly, without worrying about the CWD
ENV['PATH'] = "#{root.join('bin').to_s}#{File::PATH_SEPARATOR}#{ENV['PATH']}"


The PATH environment variable is used by Aruba to find commands we want to run. We could specify a full path in each test, but by setting PATH above we can just call the tool by its name, completely pathless, like we would be doing on a production system.

## How do you go about writing tests?

The workflow for writing stepless Aruba tests that still use the Aruba API is pretty straight forward:

1. Find the relevant step from Aruba's cucumber.rb
2. Look at how the step is implemented (what methods are called, what arguments are passed to the method, how is output captured later on, etc)
3. Take a quick look at how the method is implemented in Aruba::Api
4. Write your tests in pure-RSpec

Here's an example test:

# spec/template_spec.rb

# genud is the name of the tool we're testing
describe "genud" do
describe "YAML templates" do
it "should emit valid YAML to STDOUT" do
fqdn      = 'bprnd-test01.bulletproof.net'

# Run the command with Aruba's run_simple helper
run_simple "genud --fqdn #{fqdn} --template #{template}"

# Test the YAML can be parsed
lambda {
userdata = YAML.parse(all_output)
userdata.should_not be_nil
}.should_not raise_error
assert_exit_status(0)
end
end
end


## Multiple inputs, and DRYing up the tests

Testing multiple inputs and outputs of the tool is important for verifying the behaviour of the tool in the wild.

Specifically, we want to know the same inputs create the same outputs if we make a change to the tool, and we want to know that new inputs we add are valid in multiple use cases.

We also don't want to write test cases for each instance of test data - generating the tests automatically would be ideal.

Our first approach at doing this was to glob a bunch of test data and test the behaviour of the tool for each instance of test data:

# spec/template_spec.rb

describe "genud" do
describe "YAML templates" do
it "should emit valid YAML to STDOUT" do

# The inputs we want to test
templates = Dir.glob(root + 'templates' + "*.yaml.erb") do |template|
fqdn     = 'hello.example.org'

# Run the command with Aruba's run_simple helper
run_simple "genud --fqdn #{fqdn} --template #{template}"

# Test the YAML can be parsed
lambda {
userdata = YAML.parse(all_output)
userdata.should_not be_nil
}.should_not raise_error
assert_exit_status(0)
end
end
end
end


This worked great provided all the tests were passing, but the tests themselves became very black box when one of the test data input caused a failure.

The engineer would need to add a bunch of puts statements all over the place to determine which input was causing the failure. And even worse, early test failures mask failures in later test data.

To combat this, we DRY'd up the tests by doing the Dir.glob once in the outer scope, rather than in each test:

# spec/template_spec.rb

describe "genud" do
templates = Dir.glob(root + 'templates' + "*.yaml.erb") do |template|
describe "YAML templates" do
describe "#{File.basename(template)}" do
it "should emit valid YAML to STDOUT" do

fqdn     = 'hello.example.org'

# Run the command with Aruba's run_simple helper
run_simple "genud --fqdn #{fqdn} --template #{template}"

# Test the YAML can be parsed
lambda {
userdata = YAML.parse(all_output)
userdata.should_not be_nil
}.should_not raise_error
assert_exit_status(0)
end
end
end
end
end


This produces a nice clean test output that decouples the tests from one another while providing the engineer more insight into what test data triggered a failure:

$be rake genud YAML templates test.yaml.erb should emit valid YAML to STDOUT YAML templates test2.yaml.erb should emit valid YAML to STDOUT  ## Where to from here? The above test rig is a good first pass at meeting our goals for building systems tools: • We want to ensure our systems tools are of a consistent high quality • We want to ensure new functionality doesn't break old functionality • We want to verify we don't introduce regressions • We want to streamline peer review of changes … but we want to take it to the next level: integrating serverspec into the same test suite. Having a quick feedback loop to verify local operation of the tool is essential to engineer productivity, especially when remote operations of these type of system tools can take upwards of 10 minutes to complete. But we have to verify the output of local operation actually creates the desired service at the other end. serverspec will help us do this. ### Planet Scala #### runState for a scalaz-stream Process <status class="ok"> </status><status class="ok"> I was preparing to post this on the scalaz mailing-list but I thought that a short blog post could serve as a reference for other people as well. The following assumes that you have a good knowledge of Scalaz (at least of what's covered in my "Essence of the Iterator Pattern" post and some familiarity with the scalaz-stream library. ### My use case What I want to do is very common, just process a bunch of files! More precisely I want to (this is slightly simplified): 1. read some pipe delimited files 2. validate that the files have the proper internal structure: one(HEADER marker) one(column names) many(lines of pipe delimited values) one(TRAILER marker with total number of lines since the header) 3. output only the lines which are not markers to another file ### Scalaz stream The excellent chapter 15 of Functional Programming in Scala highlights some of the potential problems with processing files: • you need to make sure you are closing resources properly even in the face of exceptions • you want to be able to easily compose small processing functions together instead of having a gigantic loop and a bunch of variables • you want to control the amount of data that is in memory at any moment in time Based on the ideas of the book, Paul Chiusano created scalaz-stream, a library providing lots of combinators for doing this kind of input/output streaming operations (and more!). ### A state machine for the job My starting point for addressing our requirements is to devise a State object representing the both the expected file structure and the fact that some lines need to be filtered out. First of all I need to model the kind of lines I'm expecting when reading the file: sealed trait LineStatecase object ExpectHeader extends LineStatecase object ExpectHeaderColumns extends LineStatecase class ExpectLineOrTrailer(lineCount: Int = 0) extends LineState As you can see ExpectLineOrTrailer contains a counter to keep track of the number of lines seen so far. Then I need a method (referred as the State function below) to update this state when reading a new line: def lineState(line: String): State[Throwable \/ LineState, Option[String]] = State { state: Throwable \/ LineState => def t(message: String) = new Exception(message).left (state, line) match { case (\/-(ExpectHeader), HeaderLine(_)) => (ExpectHeaderColumns.right, None) case (\/-(ExpectHeaderColumns), _) => (ExpectLineOrTrailer(0).right, None) case (\/-(ExpectHeader), _) => (t("expecting a header"), None) case (\/-(ExpectLineOrTrailer(n)), HeaderLine(_)) => (t("expecting a line or a trailer"), None) case (\/-(ExpectLineOrTrailer(n)), TrailerLine(e)) => if (n == e) (ExpectHeader.right, None) else (t(s"wrong number of lines, expecting$e, got $n"), None) case (\/-(ExpectLineOrTrailer(n)), _) => (ExpectLineOrTrailer(n + 1).right, Some(line)) case (-\/(e), _) => (state, None) }} The S type parameter (in the State[S, A] type) used to keep track of the "state" is Throwable \/ LineState. I'm using the "Left" part of the disjunction to represent processing errors. The error type itself is a Throwable. Originally I was using any type E but we'll see further down why I had to use exceptions. The value type A I extract from State[S, A] is going to be Option[String] in order to output None when I encounter a marker line. This is all pretty good, functional and testable. But how can I use this state machine with a scalaz-stream Process? ### runState After much head scratching and a little help from the mailing-list (thanks Pavel!) I realized that I had to write a new driver for a Process. Something which would understand what to do with a State. Here is what I came up with: def runState[F[_], O, S, E <: Throwable, A](p: Process[F, O]) (f: O => State[E \/ S, Option[A]], initial: S) (implicit m: Monad[F], c: Catchable[F]) = { def go(cur: Process[F, O], init: S): F[Process[F, A]] = { cur match { case Halt(End) => m.point(Halt(End)) case Halt(e) => m.point(Halt(e)) case Emit(h: Seq[O], t: Process[F, O]) => { println("emitting lines here!") val state = h.toList.traverseS(f) val (newState, result) = state.run(init.right) newState.fold ( l => m.point(fail(l)), r => go(t, r).map(emitAll(result.toSeq.flatten) ++ _) ) } case Await(req, recv, fb: Process[F, O], cl: Process[F, O]) => m.bind (c.attempt(req.asInstanceOf[F[Any]])) { _.fold( { case End => go(fb, init) case e => go(cl.causedBy(e), init) }, o => go(recv.asInstanceOf[Any => Process[F ,O]](o), init)) } } } go(p, initial)} This deserves some comments :-) The idea is to recursively analyse what kind of Process we're currently dealing with: 1. if this is a Halt(End) we've terminated processing with no errors. We then return an empty Seq() in the context of F (hence the m.point operation). F is the monad that provides us input values so we can think of all the computations happening here as happening inside F (probably a scalaz.concurrent.Task when reading file lines) 2. if this is a Halt(error) we use the Catchable instance for F to instruct the input process what to do in the case of an error (probably close the file, clean up resources,...) 3. if this is an Emit(values, rest) we traverseS the list of values in memory with our State function and we use the initial value to get: 1. the state at the end of the traversal, 2. all the values returned by our State at each step of its execution. Note that the traversal will happen on all the values in memory, there won't be any short-circuiting if the State indicates an error. Also, this is important, the traverseS method is not trampolined. This means that we will get StackOverflow exceptions if the "chunks" that we are processing are too big. On the other hand we will avoid trampolining on each line so we should get good performances. If there was an error we stop all processing and return the error otherwise we emit all the values collected by the State appended to a recursive call to go 4. if this is an Await Process we attempt to read input values, with c.attempt, and use the recv function to process them. We can do that "inside the F monad" by using the bind (or flatMap) method. The resulting Process is sent to go in order to be processed with the State function Note what we do in case 2. when the newState returns an exception.left. We create a Process.fail process with the exception. This is why I used a Throwable to represent errors in the State function. Now let's see how to use this new "driver". ### Let's use it First of all, we create a test file: import scalaz.stream._import Process._val lines = """|HEADER|file |header1|header2 |val11|val12 |val21|val22 |val21|val22 |TRAILER|3""".stripMargin// save 100 times the lines above in a file(fill(100)(lines).intersperse("\n").pipe(process1.utf8Encode) .to(io.fileChunkW("target/file.dat")).run.run Then we read the file but we buffer 50 lines at the time to control our memory usage: val lines = io.linesR("target/file.dat").buffer(50) We're now ready to run the state function: // this task processes the lines with our State function// the initial State is ExpectHeader because this is what we expect the first line to beval stateTask: Task[Process[Task, String]] = runState(lines)(lineState, ExpectHeader)// this one outputs the lines to a result file// separating each line with a new line and encoding it in UTF-8val outputTask: Task[Unit] = stateTask.flatMap(_.intersperse("\n").pipe(process1.utf8Encode) .to(io.fileChunkW("target/result.dat")).run)// if the processing throws an Exception it will be retrieved hereval result: Throwable \/ Unit = task.attemptRun When we finally run the Task, the result is either ().right if we were able to read, process, and write back to disc or exception.left if there was any error in the meantime, including when checking if the file has a valid structure. The really cool thing about all of this is that we can now precisely control the amount of memory consumed during our processing by using the buffer method. In the example above we buffer 50 lines at the time then we process them in memory using traverseS. This is why I left a println statement in the runState method. I wanted to see "with my own eyes" how buffering was working. We could probably load more lines but the trade-off will then be that the stack that is consumed by traverseS will grow and that we might face StackOverflow exceptions. I haven't done yet any benchmark but I can imagine lots of different ways to optimise the whole thing for our use case. ### try { blog } finally { closing remarks } I'm only scratching the surface of the scalaz-stream library and there is still a big possibility that I completely misunderstood something obvious! First, it is important to say that you might not need to implement the runState method if you don't have complex validation requirements. There are 2 methods, chunkBy and chunkBy2, which allow to create "chunks" of lines based on a given line (for chunk) or pair of lines (for chunk2) naturally serving as "block" delimiters in the read file (for example a pair of "HEADER" followed by a "TRAILER" in my file). Second, it is not yet obvious to me if I should use ++ or fby when I'm emitting state-processed lines + "the rest" (in step 2 when doing: emitAll(result.toSeq.flatten) ++ _). The difference has to do with error/termination management (the fallback process of Await) and I'm still unclear on how/when to use this. Finally I would say that the scalaz-stream library is intriguing in terms of types. A process is Process[F[_], O] where O is the type of the output and the type of the input is... nowhere? Actually it is in the Await[F[_], A, O] constructor as a forall type. That's not all. In Await you have the type of request, F[A], a function to process elements of type A: recv: A => Process[F, O] but no way to extract or map the value A from the request to pass it to the recv method! The only way to do that is to provide an additional constraint to the "driver method" by saying, for example, that there is an implicit Monad[F] somewhere. This is the first time that I see a design where we build structures and then we give them properties when we want to use them. Very unusual. I hope this can help other people exploring the library and, who knows, some of this might end up being part of it. Let's see what Paul and others think... </status><status class="ok"> </status> ### StackOverflow #### Slick left outer join fetching whole joined row as option My join looks like this: def byIdWithImage = for { userId <- Parameters[Long] (user, image) <- Users leftJoin RemoteImages on (_.imageId === _.id) if user.id === userId } yield (user, image)  but slick fails at runtime when user.imageId is null [SlickException: Read NULL value for column RemoteImage.url] Changing the yield to } yield (user, image.?)  gives me a compile time exception, it only works on individual columns could not find implicit value for evidence parameter of type scala.slick.lifted.TypeMapper[image.type] Would there be a different way to accomplish what I'm trying to do here? (in a single query) #### Clojure Function Error "Key must be integer" I have a problem in this function although it compiles without errors! The function gets two vectors with the same length n, another vector with length 2^n, and an index. The function does a simple calculation and then returns a vector. The problem appears when trying to call the function. For example: (check [1 2 3] [1 2 3] [1 2 3 4 5 6 7 8] 1) java.lang.IllegalArgumentException: Key must be integer (NO_SOURCE_FILE:0)  Function definition: (defn check [row1 row2 arr j] ( (def x1 (nth arr (nth row1 j))) (def x2 (nth arr (nth row2 (- j 1)))) (def x3 (nth arr (nth row1 (- j 1)))) (if (<= x1 x2) ( (def row1 (assoc row1 j x3)) row1 ) ((def row1 (assoc row1 (- j 1) x2)) row1) ) ) )  #### Append Row To An HTML File I have a HTML file that contains a table with a variable amount of row. I'd like to be able to open that file, select the table, append a row (the amount of columns is fixed), and write this back to file file. I'm not quire sure how to proceed. UPDATED: Here's a simplified but HTML-valid file:  <html> <head> <title>Light Template</title> </head> <body> <div class="container"> <div class="starter-template"> <h1>CTS QA Test Report</h1> <p id="suiteintrotext"></p> <h1>Summary</h1> <table class="table" id="summarytable"> <tr> <th>#</th> <th>Test Name</th> <th>Author</th> <th>Start/End time (dur.)</th> <th>Avg. CPU%</th> <th>Pass</th> </tr> <tr> <td>one</td> <td>two</td> <td>three</td> <td>four</td> <td>five</td> <td>six</td> </tr> </table> </div> </div> </body> </html>  The second row is there to get some data for the step-by-step troubleshooting in Eclipse This is as far as I managed to go in Java: public void appendRow() throws ParserConfigurationException{ File source = new File(this.Path); Document report = null; try { report = Jsoup.parse(source, "UTF-8"); } catch (IOException e) { System.out.println("Unable to open ["+source.getAbsolutePath()+"] for parsing!"); } Elements table = report.select("#summarytable"); // Create row and td elements add append? }  My problem at this point is how to add a row, which itself should have more elements (the cells/columns). The problem right after that point would be how to save this in the HTML file? ### /r/scala #### Early bird for #scalax 2014 runs out this Friday. ### StackOverflow #### Add multiple array values to a list is possible in scala? How to add array values a & b (multiple values) to List list1  var list1:java.util.List[Array[String]]=List[Array[String]]() var a=Array[String]("Excellent","Good","Need To Improve") var b=Array[String]("1","2","3") list1.add(a) list1.add(b)  I tried this but the Array values not add to the List The array values not add to list ### TheoryOverflow #### Testbed of perfect graphs Is there a publicly available testbed of perfect graphs? Ideally, the number of vertices should be 100 or more. ### Portland Pattern Repository #### Chauncey Gardiner (by d58-111-186-52.meb802.vic.optusnet.com.au 17 hours ago) ### Lamer News #### Spring-Load Your Rails Development Environment To Speed It Up ### /r/scala #### If someone was building a distributed spatial routing system in Scala for queries across game servers. What would this be/involve ? Sorry, I'm a bit lost on this one, its a new area for me and I am trying to understand what they are doing in layman's terms. Sorry there isnt more info! Any help or hypothesis would be welcome. submitted by nebnagor [link] [comment] ### TheoryOverflow #### Rperesentation suitable for reconstruction of a tree with bounded degree I am dealing with reconstruction of molecular graphs for which unlabelled rooted trees with maximum degree 4 are fair approximations. In particular, I would like to encode a small tree (assume number of vertices is less than 20) and degree is upper bounded by 4, such that we can reconstruct the tree form the encoding, decoding is not necessary to be unique but low degeneracy is expected. Currently, I am given the number of vertices at various distance from the root, say$d = [d_1, d_2, ..., d_k]$where$d_i$is the number of vertices at distance$i$from root. However, the number of tree satisfying this condition is very large. I am looking for other representation (as an alternative to$d$) of tree which will be more suitable for reconstruction, may be approximately. I would prefer the representation which is hierarchical in the sense that, first few information are more constrained and informative, and others are gradually less so. For example, here$d_1$gives complete information on number of nodes at level 1, there is slight degeneracy in constructing level two nodes from$d_2$, etc. My focus is on approximate reconstruction rather than canonical encoding. ### StackOverflow #### clojure-variable-names for database_column_names This is a "what's most idiomatic in Clojure" question. I'm using Cassandra for my DB, with Alia as my Clojure driver (both Cassandra and Alia work phenomenally well -- couldn't be happier). The problem is this: Cassandra uses underscores (not dashes) in column names, and Clojure prefers dashes to underscores. So "user-key" in Clojure is "user_key" in Cassandra. How best to handle the mapping of Cassandra column names to Clojure variables? Because I'm using prepared statements for my CQL queries, I think the fact that column names contain underscores and not dashes is more than an implementation detail to be abstracted away -- I'm frequently putting CQL queries as strings into my Clojure code, and I think it's important to represent the CQL as it actually is. I have considered approaches that auto-magically translate dashes to underscores in query strings, so that there's a Clojure version of the CQL that gets mapped to the Cassandra version of the CQL, but this seems like an inappropriate level of abstraction. Besides you'd still need to use underscores when you ran CQL queries directly in Cassandra for troubleshooting, so you'd need to keep two different representations of column names in your head. Seems like the wrong approach. The approach I've ended up taking is to perform the mapping in a Clojure destructuring map, like this: (let [{user-key :user_key, user-name :user_name} (conn/exec-1-row-ps "select user_key,user_name from users limit 1")] )  ("conn/exec-1-row-ps" is my convenience function that just looks the CQL string up in a map, and uses the previously-prepared statement if present, or else prepares the statement and stashes it in the map, and then executes the prepared statement and returns the first row of the result set, or throws an exception if more than one row is returned). if I use the more concise {:keys []} destructuring method, then I'm stuck with underscores in my Clojure variable names: (let [{:keys [user_key user_name]} ...  That was the first approach that I tried, but it gets ugly very fast, as variable names with underscores seep through the code, and come head-to-head with ones with dashes. Confusing. Having been confronted with this issue for a long time, doing the conversion in the destructuring map, where Clojure "variable-name" and Cassandra "column_name" are side-by-side feels like the best solution. It also lets me expand out short_col_nms to more-descriptive-variable-names when I want to. This bears some resemblance to the mapping that Clojure does of underscores in filenames to dashes in namespaces, so it feels like there is some precedent for doing the mapping like this. In the filename/namespace case, Clojure does the mapping automagically, and so perhaps the direct analog would be a version of {:keys []} destructuring that mapped dashes to underscores. I'm a relative newbie to Clojure, so I realize there may be better ways to do this. Hence my question. One improvement that I've considered is writing a macro that builds the destructuring map dynamically at compile-time. But I don't know how to write a macro that operates that early in the compilation process. #### object field import in scala I have a very simple question of style. What is the best way to access a field or a method defined in an object ? For example if I have an object Foo that define a field bar, would you rather do import Foob.bar and then refer to it in the scope as bar or call Foo.bar wherever you need the field. From what I have read in most libraries, Foo.bar seems the standard way to do it but I would like to be sure. Thanks, #### java.lang.IncompatibleClassChangeError: Found class scala.collection.mutable.ArrayOps, but interface was expected I am trying to run a Scala jar compiled in version 2.10.2 but am getting the following error. I am presuming it is a problem with the library versions but am having no luck:  java.lang.IncompatibleClassChangeError: Found class scala.collection.mutable.ArrayOps, but interface was expected at scala.concurrent.duration.Duration$.scala$concurrent$duration$Duration$$words(Duration.scala:72) at scala.concurrent.duration.Duration.scalaconcurrentdurationDuration$$expandLabels(Duration.scala:74) at scala.concurrent.duration.Duration$$anonfun4.apply(Duration.scala:93) at scala.concurrent.duration.Duration$$anonfun$4.apply(Duration.scala:93)
at scala.collection.TraversableLike$$anonfunflatMap1.apply(TraversableLike.scala:239) at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59) at scala.collection.immutable.List.foreach(List.scala:76) at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239)
at scala.collection.immutable.List.flatMap(List.scala:76)
at scala.concurrent.duration.Duration$.<init>(Duration.scala:93) at scala.concurrent.duration.Duration$.<clinit>(Duration.scala)
at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:139) at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:470) at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:93) at com.bullionvault.bvbot.BVBots.<init>(BVBots.scala:22) at com.bullionvault.bvbot.BVBots$$anon1.<init>(BVBots.scala:13) at com.bullionvault.bvbot.BVBots.main(BVBots.scala:13) at com.bullionvault.bvbot.BVBots.main(BVBots.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at scala.tools.nsc.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:78) at scala.tools.nsc.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:24)
at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:88) at scala.tools.nsc.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:78)
at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:101) at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:33)
at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:40) at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:56)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:80)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:89) at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)  ### TheoryOverflow #### NFA Algorithms in the literature for Micron's new Automata Processor Micron just announced they are shipping a fundamentally new architecture that intermingles compute circuits with RAM, http://www.micron.com/about/innovations/automata-processing What are some of the common DFA based algorithms that will quickly fall out of favor for their NFA counterparts in the literature? ### Fred Wilson #### Maintaining High Maintenance The New Yorker gives high praise for this web video series called High Maintenance. The episodes are short (~5min) and there are eleven episodes in all. It is about a pot dealer who rides around NYC and meets all sorts of strange and outrageous people. This bit from the lead actor in the New Yorker piece rings true to me after I watched a bunch of episodes last night: The thing about weed is, we didn’t want to use it as a punch line. Instead, it’s this substance that, like chocolate, causes people to expose their own foibles. People become so human in pursuit of this thing. And the interaction they have with the person bringing it is often tragic, because there are a lot of lonely people out there who order it and then that is their human interaction for the day. But here's the part of the New Yorker article that has me thinking outloud this morning. When I spoke with Sinclair and Blichfeld recently, they were on the West Coast signing a script deal with a major network, newly on the path to converting “High Maintenance” into a full-length cable show. It's a bit upsetting to me that the "major leagues" for filmmakers, writers, and actors who make it on the web is still the cable business. Why can't entrepreneurs build something that will work better for emerging web filmmakers than that? We have investments in Kickstarter and VHX, both of which are changing the game for filmmakers. We are also big fans of Vimeo, where High Maintenance is hosted. But this High Maintenance story tells me that we haven't yet built enough technology, distribution, and monetization systems so that filmmakers can be truly independent and realize their vision and have the financial sucess that should come with great work. So there is more to do here. ### CompsciOverflow #### What does the symbol # mean when it comes to languages Given the following: $$\{ w\#x | w^R \text{ is a substring of x, with x and w \in \Sigma^*} \}$$ What does$w\#x$denote? ### StackOverflow #### DeDuplication error with SBT assembly plugin I am trying to create an executable jar using SBT assembly plugin. I am ending up with below error : [error] (app/*:assembly) deduplicate: different file contents found in the following: [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-continuation/jars/jetty-continuation-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-http/jars/jetty-http-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-io/jars/jetty-io-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-security/jars/jetty-security-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-server/jars/jetty-server-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-servlet/jars/jetty-servlet-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-util/jars/jetty-util-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-webapp/jars/jetty-webapp-8.1.8.v20121106.jar:about.html [error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-xml/jars/jetty-xml-8.1.8.v20121106.jar:about.html [error] Total time: 2562 s, completed Dec 5, 2013 12:03:25 PM  After reading wikis of assembly plugin, i have added merge strategy in build.scala file. Seems it is not working. I am not sure whether it is right fix or not. Can someone suggest me the right strategy. Below is the code which i have in build.scala file : mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) => { case "about.html" => MergeStrategy.discard case "logback.xml" => MergeStrategy.first //case PathList("logback.xml") => MergeStrategy.discard case x => old(x) } }  I have coded plugin integration with my app as per this doc : Standalone deployment of Scalatra servlet I tried diffrent strategies like MergeStrategy.rename and MergeStrategy.deduplicate. But nothing works.. Looking for help... ### TheoryOverflow #### Shifted geometric means Reading papers on the runtime comparison of two different algorithms on a given benchmark set, one often reads the term "shifted geometric means". Could anyone explain how it is defined and why it is a good idea to use instead of, e.g., arithmetic or geometric means? ### CompsciOverflow #### Right moving turing machine I am interested in simulating any turing machine with a turing machine that is allowed only to move right. I guess that it should be pretty standard material and likely it is trivial (or known to be false). Does anyone have a pointer to a reference? ### StackOverflow #### Continue reading stdin until "END" in Scala? How do I continue reading stdin until "END" string reached, in Scala? Here's what I've tried: val text = Iterator.continually(Console.readLine).takeWhile(_ != "END").toString  ### CompsciOverflow #### Why is this not a regular language So I recently had a problem where I had to create a regular language. After consulting my professor on my solution he told me it was close to correct but to check my definition of a regular language. I am somewhat lost on why my solution is not regular? From my understanding it meets the requirements of being a right regular language since there is only one non-terminal on the right side of the expression and the non-terminal is the rightmost symbol on the right side of the expression. Are pipe symbols not allowed? ### StackOverflow #### What is the difference between :while and :when in clojure? I'm studying clojure but not quite clear on the difference between the :while and :when test: => (for [x [1 2 3] y [1 2 3] :while (= (mod x y) 0)] [x y]) ([1 1] [2 1] [2 2] [3 1]) => (for [x [1 2 3] y [1 2 3] :when (= (mod x y) 0)] [x y]) ([1 1] [2 1] [2 2] [3 1] [3 3])  Can anybody help by elaborating on them ? ### CompsciOverflow #### Turing machine for$a^i b^j$with$i \geq j$I would have a brief question about how to construct a Turing machine that is accepting only this language:$\qquad\displaystyle L_2 = \{a^i b^j \mid i \geq j \}$. I can't come up with any mechanism that would preserve that there are greater or equal number of 'a's than 'b's. Could you suggest me something? ### StackOverflow #### How to construct wildcard type in Scala macro? I am unable to find out how to programatically construct existential types in Scala macros. For example, let's assume that I have a ClassSymbol that represents a class C[T] that has one type parameter. Now, how do I programatically construct the type C[_ <: java.lang.Number] ? In particular, I have no idea how to use the ExistentialType constructor object. Looking at its signature: def apply(quantified: List[Symbol], underlying: Type): ExistentialType  What do I pass as quantified? #### Scala: Syntax of passing an execution context to Future's map function A previous version of this question did not come to the point, so I tried to boil it down: The map function of a Future takes an ExecutionContext, as shown below (taken of API 2.10.3) def map[S](f: (T) ⇒ S)(implicit executor: ExecutionContext): Future[S]  I want to pass my own ExecutionContext, but I do not succeed. It seems as if I do not get the syntax right. How can I pass my ExecutionContext to a map function of a Future? What I have got is: val executorService = Executors.newFixedThreadPool(9) val executionContext = ExecutionContext.fromExecutorService(executorService) def myFunction(mt : MyType): Unit = println("my function") def getSomeFuture()(executor: ExecutionContext): Future[MyType] = {..} // function is served by my execution context. A Future is returned. val f = getSomeFuture()(executionContext) // compiles if I provide the standard global execution context - sth. I do NOT want f map (myFunction) // does not compile f map (myFunction)(executionContext) // does not compile f map (item => println("sth."))(executionContext)  #### How to communicate between ZeroMQ contexts in different Goroutines? I'm using this as boilerplate, except that in the same program I also have some goroutines that are the workers and connect to the backend endpoint, tcp://127.0.0.1:5560. What I would like to do is have it connect through a more efficient way, such as ipc://, inproc://, or even unix sockets. I've tried those, and it didn't work. Channels is a no-go with ZeroMQ right ? So how do I connect different goroutines with ZeroMQ contexts, without tcp ? Is there a better alternative ? update: The code: // Simple message queuing broker // Same as request-reply broker but using QUEUE device // // Author: Brendan Mc. // Requires: http://github.com/alecthomas/gozmq package main import ( zmq "github.com/alecthomas/gozmq" ) func startWorker() { context, _ := zmq.NewContext() defer context.Close() worker, _ := context.NewSocket(zmq.REP) //err := worker.Connect("ipc:///backend") // Tried it, but nothing //err := worker.Connect("inproc:///backend") // Tried it, but nothing err := worker.Connect("tcp://127.0.0.1:5560") // this works if err != nil { fmt.Println(err) } for { data, err := worker.Recv(0) fmt.Println(string(data)) worker.Send([]byte("I got your data"), 0) } } func main() { context, _ := zmq.NewContext() defer context.Close() // Socket facing clients frontend, _ := context.NewSocket(zmq.ROUTER) defer frontend.Close() frontend.Bind("tcp://*:5559") // Socket facing services backend, _ := context.NewSocket(zmq.DEALER) defer backend.Close() //backend.Bind("ipc:///backend") // Tried it, but nothing //backend.Bind("inproc:///backend") // Tried it, but nothing backend.Bind("tcp://*:5560") // this works for i := 0; i < 4; i++ { go startWorker() // Start workers in a separate goroutine } // Start built-in device zmq.Device(zmq.QUEUE, frontend, backend) // We never get here… }  #### How can I use 'newsyslog' to trigger a script everytime my log file is rolled over or rotated? How can I use 'newsyslog' to trigger a script everytime my log file is rolled over or rotated? The script is a parser which collects the important information from the log and archives it. Is using the path_to_pid_cmd_file in newsyslog.conf the only way to do something like this? The problem I have with this approach is that this means my parser script would need to always be running/sleeping in the background and I would be signalling it. I was wondering if there was a cleaner way to do this? Just before the logfile is to be rotated, the parser script should be run and data archived after which the parser dies. And this happens each time a log file is rotated. I did hear about 'logrotate' supporting functionality like this with the pre-rotate handler but is there a way to accomplish this with 'newsyslog' or 'syslog'. ### CompsciOverflow #### Request for information on the invited talk of "Challenges for Formal Methods in Distributed Computing" by Jennifer Welch I am interested in the invited talk titled "Challenges for Formal Methods in Distributed Computing" by Jennifer Welch given at CAV'2013. However, I found nothing but the title itself through the following ways: 1. googling with the title as key words; 2. visiting the homepage of the author Jennifer Welch; 3. through DBLP. So, any more information will be appreciated. ### StackOverflow #### Play Framework: how to refer to a template in another sub-project Given the following multi-project Play application... + myApp + app + conf + modules + myModule1 | + app | | + controllers | | + models | | + views | | + main.scala.html | + conf + myModule2 + app | + controllers | + models | + views | + index.scala.html + conf  ... how do I call myModule1/app/views/main.scala.html from myModule2/app/views/index.scala.html? Is it possible to define a layout template once and then reuse it in all sub-projects? #### reduce-while function in clojure? I am new to Clojure programming, and would like to know what is the idiomatic way to do the following thing: 1. I would like to sum a collection of numbers nums, which may contains a large number of numbers, let's assume there are only positive numbers. 2. I don't care the exact sum if the sum is very large. For example, if the sum of the numbers is larger than 9999, I would simply return 10000 without summing the remaining numbers at all. If I implement it with some OO language such as Java, I may do it like below: private int sum(int[] nums) { int sum = 0; for(int n : nums) { if(sum > 9999) { sum = 10000; break; } else { sum += n; } } return sum; }  A naive implementation in Clojure may look like: (let [sum (reduce + nums)] (if (> sum 9999) 10000 sum))  However, this seems to waste some CPU resource to sum the entire collection of numbers, which is not desired. I am looking for something like take-while function but for reduce, but cannot find it. Is there something like: (reduce-while pred f val coll)  Or is there any other Clojure idiomatic way to solve this problem? I think the solution can be applied to a set of problems requiring similar logic. Any comment is appreciated. Thanks. #### Dynamic scope in macros Is there a clean way of implementing dynamic scope that will "reach" into macro calls? Perhaps more importantly, even if there is, should it be avoided? Here's what I'm seeing in a REPL: user> (def ^:dynamic *a* nil) > #'user/*a* user> (defn f-get-a [] *a*) > #'user/f-get-a user> (defmacro m-get-a [] *a*) > #'user/m-get-a user> (binding [*a* "boop"] (f-get-a)) > "boop" user> (binding [*a* "boop"] (m-get-a)) > nil  This m-get-a macro isn't my actual goal, it's just a boiled down version of the problem I have been running into. It took me a while to realize, though, because I kept debugging with macroexpand, which makes everything seem fine: user> (binding [*a* "boop"] (macroexpand '(m-get-a))) > "boop"  Doing macroexpand-all (used from clojure.walk) on the outer binding call leads me to believe that the "issue" (or feature, as the case may be) is that (m-get-a) is getting evaluated before the dynamic binding takes: user> (macroexpand-all '(binding [*a* "boop"] (f-get-a))) > (let* [] (clojure.core/push-thread-bindings (clojure.core/hash-map #'*a* "boop")) (try (f-get-a) (finally (clojure.core/pop-thread-bindings)))) user> (macroexpand-all '(binding [*a* "boop"] (m-get-a))) > (let* [] (clojure.core/push-thread-bindings (clojure.core/hash-map #'*a* "boop")) (try nil (finally (clojure.core/pop-thread-bindings))))  Here's my crack at a workaround: (defmacro macro-binding [binding-vec expr] (let [binding-map (reduce (fn [m [symb value]] (assoc m (resolve symb) value)) {} (partition 2 binding-vec))] (push-thread-bindings binding-map) (try (macroexpand expr) (finally (pop-thread-bindings)))))  It will evaluate a single macro expression with the relevant dynamic bindings. But I don't like using macroexpand in a macro, that just seems wrong. It also seems wrong to resolve symbols in a macro--it feels like a half-assed eval. Ultimately, I'm writing a relatively lightweight interpreter for a "language" called qgame, and I'd like the ability to define some dynamic rendering function outside of the context of the interpreter execution stuff. The rendering function can perform some visualization of sequential instruction calls and intermediate states. I was using macros to handle the interpreter execution stuff. As of now, I've actually switched to using no macros at all, and also I have the renderer function as an argument to my execution function. It honestly seems way simpler that way, anyways. But I'm still curious. Is this an intended feature of Clojure, that macros don't have access to dynamic bindings? Is it possible to work around it anyways (without resorting to dark magic)? What are the risks of doing so? ### /r/scala #### Scala vs Idris: Dependent Types, Now and in the Future ### QuantOverflow #### Large (5K-10K) non positive definite (particularly near singular) covariance matrices and treatments for Cholesky decomposition I have a very large covariance matrix (around 10000x10000) of returns, which is constructed using a sample size of 1000 for 10000 variables. My goal is to perform a (good-looking) Cholesky decomposition of this matrix. However, as expected, this matrix is near-singular with very small ( < 10^-10 ) eigenvalues (around 5K-6K out of 10K). I tried a naive approach; doing an eigenvalue decomposition, and setting all the eigenvalues that are less than 10^-10 to 10^-10. In other words, if eigenvalue < 10^-10, then eigenvalue = 10^-10. After that, I reconstructed the matrix with the modified eigenvalue matrix. However, although I can perform Cholesky decomposition, it is very unstable. What is the best way to handle this PD approximation for large matrices? ### Wondermark #### New cards + the 2014 Wondermark Calendar. I have some new holiday cards for you! The line of Wondermark greeting cards — Monocle Poppers™ — are always popular around this time of year, and I’m pleased to share these 2014 DEBUT DESIGNS: “Mad Night” “Antlers” The design above is by the wonderfully talented Emily Partridge! Emily gave me this design last year and told me I could do what I wanted with it, and what I want to do is pay her a royalty on these cards. Because they’re super great and I like ‘em! These and dozens of other card designs are available for you to order right now, if you like! ### The 2014 Wondermark Calendar Is available now for pre-order! Its title is: Roll-A-Sketch Yearbook: 2014 Graduating Class A Record; A Reminiscence; A Catalog; A Chronicle; An Indictment; A Regret; A Cautionary Tale For All Time of The Grand Experiment & Mistake. LIKE EVERY YEAR SINCE 2008 — it is available in strictly limited edition only, each piece individually signed and numbered. LIKE EVERY YEAR SINCE 2009 — you can get the calendar with display apparatus included, or just the cards themselves, if you want to reuse the stand from last year. LIKE LAST YEAR — this is a biweekly progressive calendar, meaning rather than being broken into months, it’s broken into 14-day chunks, and no matter what day of the month it is, you can always see at least two weeks ahead. (More on the unique design in these posts from last year: Part 1 / Part 2 / Part 3) LIKE LAST YEAR — The art is created by me, and beautifully watercolored by my friend Max Loren Shepard! ### New For This Year The art on the calendar is Roll-a-Sketch drawings, with added biographical information about each strange creature! The art is all mine, and the color is by Max. Last year, we ordered wooden backboards cut to size and then had to spend a bunch of time sanding and drilling holes in every single board. This year, however, I’ve made the acquaintance of a man with a laser cutter! (Seen here working on the deluxe editions of the Machine of Death game.) The backboards available this year are custom laser-cut wood, with integrated easel legs, manufactured to order by my new best friend, Jason Lioi of Dapper Devil. The separate metal easel is no longer required; this backboard comes in three pieces and can stand on its own once assembled (which takes all of two seconds). Of course, the cards themselves are fully backwards compatible with last year’s stands. In addition, I am offering — as an exclusive bonus with the calendars — a brand new book, called Horrid Little Stories: Horrid Little Stories collects all the calendar content (art and text both — not the grids, though, because who cares) from the 2008-2012 Wondermark calendars. Sixty grim little tales in all! (Max painted the cover to this one too.) The book is available as an option with the calendar, if you like! All copies of the book that ship with the calendar will be signed by me. For the moment, this particular book is only available here, now, with the calendar. OH YES AND THIS IS ALSO IMPORTANT I still have all of last year’s original paintings from the Gaxian Almanac. They look like this: They’re super wonderful pieces of art — my halfway-okay pencil drawings were really enlivened by Max’s beautiful paintings! So this year, I’m offering the ultra-limited ART COLLECTOR BUNDLE, which gets you the new 2014 calendar, the Horrid Little Stories book, and one of these pieces of original art, matted with that card’s text from last year’s calendar, and signed by me and Max. The overall matted pieces are 8×10″. Only 29 Art Collector Bundles are available! And only 250 calendars overall are available! This is a PRE-ORDER. The calendars, books, and original art will ship by December 18. (If you order greeting cards and a calendar both, unless you specify otherwise I will ship you the cards right away, then follow up with the calendar later in a separate shipment.) THE 2014 WONDERMARK CALENDAR: AVAILABLE NOW. Enliven your life in 2014 with 27 fortnights of Wondermark and Roll-a-Sketch!! ### UnixOverflow #### Why is it needed to erase the first MByte of an OpenBSD CRYPTO device? The following command, executed from the command line, would configure the device softraid0 with one special device (/dev/sd2e) and an encrypting volume: # bioctl -c C -l /dev/sd2e softraid0 bioctl will ask for a passphrase, which will be needed to unlock the encrypted disk. After creating a newly encrypted disk, the first megabyte of it should be zeroed, so tools like fdisk(8) or disklabel(8) don't get confused by the random data that appears on the new disk. This can be done with the following command (assuming the new disk is sd3): # dd if=/dev/zero of=/dev/rsd3c bs=1m count=1  My question: but exactly why is it needed to erase the first MByte of the CRYPTO device? Are there any disadvantages if I miss doing this? ### /r/netsec #### LAYER 7 DDOS ATTACKS: DETECTION & MITIGATION submitted by MiM7 [link] [comment] ### CompsciOverflow #### array with a twist Imagine that we have an array like structure A with n elements all of which are initially 0. ($A[i]=0$) What is a data structure that supports the following operations: 1) Given an element A[i]=0 set A[i] to 1 in O(1) worst case 2) Given the index i return A[i] in O(1) worst case 3) Given the index i retrun the smallest$j\geq i$such that$A[j]=0$or$-1$if there is no such index in amortized time as small as possible Obviously an array supports the first 2. It doesnt not support 3). How to modify it? ### Planet Emacsen #### Ben Simon: So Smart: The Galaxy S3 as Dumb Terminal Before I went off to college and started my Computer Science degree I was such a naive little computer geek wannabe. I thought I knew the big rivalries: Microsoft Word vs. Word Perfect, Excel vs. Lotus and of course, Windows vs. Mac. As I got indoctrinated in the ways of Computer Science, I quickly learned that these were all but petty squabbles. The true debate was Emacs vs. Vi, and most importantly, there was no real operating system debate: you used Unix and you scoffed at everything else. After all, these new computers on campus were called "Work Stations" for a reason; real stuff got done there. The computer I proudly brought from home featuring the latest Windows 95 OS (or was it 3.1?) was little more than a toy when compared to the green screen terminals powered by Unix that were sprinkled throughout campus. I promise, this trip down memory lane is going somewhere useful. Learning Unix meant re-learning nearly everything I thought I knew about computers. I found myself asking a simple question over and over again: I know how do to task X on a Windows box, but how would I do it using little more than the Unix command line? For example, how does one format a document without Microsoft Word? (answer: TeX). Or work with numbers without Excel? (answer: awk and/or sc). Or read email without Eudora? (answer: emacs and Rmail). Often, I learned that these command line tools weren't just acceptable replacements for the Windows program I was use to, but had features that would blow them away. TeX didn't just format text, it made my math homework look like it had been professionally typeset. What does all this have to do with the modern age? Especially, in an age when I use all these same command line tools while still running Windows. Well, my use-my-Galaxy-S3-as-a-travel-laptop-replacement-device effort is actually still going pretty strong. On weekends, when I'd usually bust out my netbook to do any work that needs to be done, I've been instead hunching over my Galaxy S3 and bluetooth keyboard. And while some of the tasks I'm tackling are taking longer to complete than on a standard laptop, I'm still quite amazed at what I can get done on this little Android device (like buying Chanukah presents, posting blog entries and reviewing hundreds of photos from my DSLR). While the S3 is a remarkable computer (1.51Ghz, dual core with 1.5Gig of RAM - stats that were completely unimaginable in a desktop computer when I was in college), it has a number of high quality ssh clients which let me connect up to any Linux box on the net. These days I'm using JuiceSSH to do this, which works well with my external keyboard. And once a connection has been established, I gain access to all the computer power of that remote device. Yes, I've recreated the ultra powerful green screen dumb terminals of my college days. But more importantly than that, I find myself asking that same question I used to ask all the time: how would I tackle X without a Windows, only using command line tools? Consider this example: how would I manage Picasa albums or Blogger posts from the command line? Impossible? Not at all. Just use GoogleCL, the Google Command line tool set. I ssh'd to one of my servers, ran sudo yum install googlecl and a few moments later I had a remarkable level of access to these Google services from nearly any ssh capable device. It sure is fun to be back to discovering command line tools I didn't know existed, yet have immense power. I'm not sure how much longer my Phone As Laptop replacement (or, device convergence as my Dad so appropriately calls it) experiment is going to go on for, but it's expanding my horizons, which is always a good thing. ### /r/netsec #### Bypassing Content Security Policy with Java Applets ### /r/scala #### Start With Akka: Let’s Write a Scraper ### StackOverflow #### Function Equality Notion Is there any notion of function equality in Scala? scala> def foo(x: Int) = x + 1 foo: (x: Int)Int scala> def bar(x: Int) = x + 1 bar: (x: Int)Int  So I can't call == on a function. scala> foo == bar <console>:10: error: missing arguments for method foo; follow this method with _' if you want to treat it as a partially applied function foo == bar ^  Besides verifying that foo(x) == bar(x) for all x's (not sure how to do that), how else can I verify equality? #### scala, transform a callback function to an iterator/list For the sake of this question, I have the following example code which I cannot change:  trait Receiver[T] { def receive(entry: T); def close(); } def f1(r: Receiver[Int]) { new Thread() { override def run() { (0 to 9).foreach { i => r.receive(i); Thread.sleep(1000); }; r.close(); }}.start(); }  that is, f1 takes a Receiver, calls receive(i) 10 times and then closes. (that is an example, f1 could also call receive(i) infinite times) so, one implementation of Receiver could be:  class MyReceiver extends Receiver[Int] { def receive(entry: Int) { if (entry % 2 == 0) { println("Entry#" + entry); } } def close() {} }  and I would call it like this:  f1(new MyReceiver());  now, the question is: without changing the f1 and Receiver, how can I use an iterator or list approach, instead of filtering and mapping inside MyReceiver.receive function? I've implemented the following, and it works. However I guess that there is a simpler way to achieve this.  import java.util.concurrent.Semaphore class ReceiverToIterator[T] extends Receiver[T] with Iterator[T] { var lastEntry: T = _ var waitingToReceive = new Semaphore(1) var waitingToBeConsumed = new Semaphore(1) var eof = false waitingToReceive.acquire() def receive(entry: T) { println("ReceiverToIterator.receive(" + entry + "). START.") waitingToBeConsumed.acquire() lastEntry = entry waitingToReceive.release() println("ReceiverToIterator.receive(" + entry + "). END.") } def close() { println("ReceiverToIterator.close().") eof = true waitingToReceive.release() } def hasNext = { println("ReceiverToIterator.hasNext().START.") waitingToReceive.acquire() waitingToReceive.release() println("ReceiverToIterator.hasNext().END.") !eof } def next = { println("ReceiverToIterator.next().START.") waitingToReceive.acquire() if (eof) { throw new NoSuchElementException } val entryToReturn = lastEntry waitingToBeConsumed.release() println("ReceiverToIterator.next().END.") entryToReturn } }  so, now I can use the filter, map and foreach implementations of the iterator:  val receiverToIterator = new ReceiverToIterator[Int](); f2(receiverToIterator); receiverToIterator.filter(_ % 2 == 0).map("Entry#" + _).foreach(println);  So, the question is, without changing f1 and Receiver, how can I get a simpler and nicer implementation of ReceiverToIterator? ps: note that f1 could also call receive(i) infinite times, so it is not an option to collect everything in a list. also, it shouldn't keep all the results in memory. Update: Current solution Current solution using the BlockingQueue[Option[T]] as suggested by nadavwr As I find this callback to iterator pattern quite often, I've implemented to be reusable like this:  trait OptionNextToIterator[T] extends Iterator[T] { def getOptionNext: Option[T]; var answerReady: Boolean = false var eof: Boolean = false var element: T = _ def hasNext = { prepareNextAnswerIfNecessary !eof } def next = { prepareNextAnswerIfNecessary if (eof) { throw new NoSuchElementException } val retVal = element answerReady = false retVal } def prepareNextAnswerIfNecessary { if (answerReady) { return } synchronized { getOptionNext match { case None => eof = true case Some(e) => element = e } answerReady = true } } } trait QueueToIterator[T] extends OptionNextToIterator[T] { val queueCapacity: Int val queue = new ArrayBlockingQueue[Option[T]](queueCapacity) var queueClosed = false def queuePut(entry: T) { if (queueClosed) { throw new IllegalStateException("The queue has already been closed."); } queue.put(Some(entry)) } def queueClose() { queueClosed = true queue.put(None) } def getOptionNext = queue.take }  now, I can implement the question of this post like this:  class ReceiverToIterator[T](val queueCapacity: Int = 1) extends Receiver[T] with QueueToIterator[T] { def receive(entry: T) { queuePut(entry) } def close() { queueClose() } // other things if necessary } val receiverToIterator = new ReceiverToIterator[Int](queueCapacity = 3); f1(receiverToIterator); Thread.sleep(3000) // test that f1 is not blocked before 3 calls at this point receiverToIterator.filter(_ % 2 == 0).map("Entry#" + _).foreach(println)  but I still think that there should be a simpler solution. I needed to split the implementation into two traits, one that implements getOptionNext (instead of next() and hasNext()), and OptionNextToIterator which transforms this to a real iterator. that's because BlockingQueue lacks a method that "Retrieves, but does not remove, the head of this queue, waiting if necessary until an element becomes available." (as the take() version of pool() but for peek()). also, I am using the synchronized block inside OptionNextToIterator, which I don't like very much. So, is there a simpler and nicer version to implement this? (without using Iteratees, I will study this option later, and open a new stackoverflow question if necessary) ### QuantOverflow #### What is the difference between Option Adjusted Spread (OAS) and Z-spread? I am preparing for the CFA level 2 exam, I got confused by the concept Z-spread and OAS. When a call option is added to a bond, since it is not favorable to the bond buyer, they would require more spread (which is the OAS) for this instrument in order to get more discount on the bond price. So to me, Z spread should be less than the OAS. But this is not what has been discussed in the book. Can someone help with this? ### StackOverflow #### Can I enforce one function as both right associate and left associate in scala? I'd like define scalar multiply vector just as following: Vector * scalar (1) scalar * Vector (2)  I could define * as the function of class Vector which implement the first case. Then how can I implement the second case, one way I can think of is using the right associate function, but it require me to add "::" in the function name which is not readable. I want the second case just looks like the first case. How can I make it in scala ? Thanks ### TheoryOverflow #### Let A be the set of all strings of the form www(with binary alphabet). Show that A is decidable in linear time and that A is in L I am learning the Computer theory, this question is related to the course. ### StackOverflow #### Populate an array with matrix elements from a file in Scala How do I populate an array with matrix elements from a file in Scala? I know that  for (val line<- Source.fromFile("mask.txt").getLines){ }  will fetch the elements row by row but how do I split each row into individual elements and store them as elements of an array? ### TheoryOverflow #### Dynamic 2-dimensional orthogonal range reporting in external memory and linear space Orthogonal 2-dimensional range reporting is the problem of storing a set of values from$U \times V$, where$U$and$V$are totally ordered universes, subject to queries of the form "Return all stored points in the four-sided box$[a,b] \times [c,d]$", where$a,b \in U$and$c,d \in V$are parameters of the query. Nekrich, in "Orthogonal range searching in linear and almost-linear space ", summarizes the best known results for linear space: $$\begin{array}{|l|l|l|} \hline \text{Reference} & \text{Query Time} & \text{Modification Time} \\ \hline \text{Kreveld and Overmars} & O(\sqrt{n \lg n} + k) & O(\lg n)\\ \hline \text{Chazelle} & O((k+1)\lg^2 n) & O(\lg^2 n)\\ \hline \text{Nekrich} & O(\lg n + k \lg^\varepsilon n) & O(\lg^{3+\varepsilon} n) \\ \hline \end{array}$$ What are the best known extensions of these in external memory? The result from Arge et al.'s "On two-dimensional indexability and optimal range search indexing" uses superlinear space. The only generalizations I know have bounds $$\begin{array}{|l|l|l|} \hline \text{Reference} & \text{Query Time} & \text{Modification Time} \\ \hline \text{Kanth and Singh} & O(\sqrt{n/B} + k/B) & O(\lg_B n)\\ \hline \text{Procopiuc et al.} & O(\sqrt{n/B} + k/B) & O\left(\frac{1}{B} \left( \lg_{M/B} \frac{n}{B}\right) \left(\lg \frac{n}{M}\right)\right)\\ \hline \end{array}$$ ### StackOverflow #### "email:Pk[String]" vs "id:Pk[Long]" In some examples, I'm seeing email:Pk[String] under the User class but in others I see id:Pk[Long] (both not assigned). A user can change their email as much as they want but the ID will not change so is there a (security) reason behind this? ### TheoryOverflow #### If a ∈ P and b is nontrivial , then b ∩ a ≤ b [on hold] If a ∈ P and b is nontrivial , then b ∩ a ≤ b. Hello I was wonder if I am approaching this question correctly. My intuition is that since a is part of polynomial and we are taking the intersection between a and b, therefore the result has to be part of b. ### DragonFly BSD Digest #### Book Review: Perl One-Liners No Starch Press noticed that I keep talking about Michael W. Lucas’s BSD-related books, and I’ve linked to Peteris Krumins’s catonmat site before, so they sent a copy of Krumins’s new “Perl One-Liners” book to me. Here’s the hook for me: Perl was the first language I wrote a program of any real use in. Years ago, I had the Perl Cookbook. It was a pretty simple formula, where I’d have a problem. I’d look it up in the Perl Cookbook. If there was already a recipe that matched what I needed, I was set. I ended up having to stuff the book into a binder because the spine broke. This reference is essentially what the Perl One-Liners book is, though this is less about programming and more about the solution you need right now. The book realizes this and it’s laid out like a menu. Flip through the index to find your problem, and then type the answer. The book even includes a link to a text file that you can copy down and grep for answers – I won’t link to it because it’s not mentioned on the author’s page, though he does include example chapters. It’s not about learning Perl, and it’s not about technique – these are one-liners, after all. If you are doing the sort of thing Perl excels at, like text mangling, this will be a book full of tools for you. I think the author is going to continue in this style; he’s done a lot of one-liner articles and even some previous e-books. Probably a good idea to make this disclaimer: As with other books, I get no reward for this review, unless you count me having another book in the house. That’s more of a problem than a benefit for me. ### /r/netsec #### Hacking the DIR-100 rev. D1 ### StackOverflow #### How to get object from Play cache (scala) How to get object from Play cache (scala) Code to set: play.api.cache.Cache.set("mykey98", new Product(98), 0)  Code to get: val product1: Option[Any] = play.api.cache.Cache.get("mykey98")  I get Option object. How to get actual Product object I stored in first step. ### CompsciOverflow #### LL(1) parsing table of left-recursive grammar I have been taking a compiler course of Stanford on coursera. I have a slight misunderstanding on the parsing table of the following grammar: S -> Sa | b According to the professor parsing table looks like this: • If the leftmost nonterminal is S and the input is 'a', the production is nothing. • If the leftmost nonterminal is S and the input is 'b', the production is both Sa and b. So if input is just b, then it is correct, it will go directly to production b. But how will it go to production Sa for input b? ### /r/compsci #### Can anyone help me understand how to derive and follow these Phrase-Structure Grammars? http://i39.tinypic.com/wqq1ci.png Especially the second one. And is it necessary to have the production rule S-->A in the first question? Or could you treat S as A, and eliminate A as redundant? submitted by Grammars7i [link] [1 comment] ### /r/netsec ### CompsciOverflow #### mpi row-wise matrix multiplication Hello guys i am doing this program where i am supposed to multiply two square matrices of the same size using mpi. how this work is that each process would take care of a n/p rows and n cols in mat_C. what i am doing is multiplying ((id-1+p)%p)*r/p in mat_A by every col in mat_B and then switching mat_C thought out the rest of the process. I dont if i make myself clear but here is the code where i do the multiplication and then send mat_B to the other process in form of a ring. j = id*(r/p); for(i=0; i<p; i++) { /* calculate the partial sum for matC given the row band of A and B (see notes on row-wise matrix multiplication). */ //printf("here proc = %d\n\n", id); for(m = 0; m < r/p; m++){ for(k = 0; k < r/p; k++){ for(n = 0; n < r; n++){ aux_matC[k][n] += aux_matA[k][j+m] * aux_matB[m][n]; } } } j = ((id-1+p)%p)*r/p; if(i < p-1) { /* mpi send storage_matB to the next process (id+1)%p */ /* mpi receive storage_matB from the previous process */ MPI_Sendrecv(storage_matB, r*(r/p), MPI_FLOAT, (id+1)%p, 1, storage_matB, r*(r/p), MPI_FLOAT, (id-1+p)%p, 1, MPI_COMM_WORLD, MPI_STATUS_IGNORE); for(x=0; x<r/p; x++) aux_matB[x] = &storage_matB[x*r]; }  } lets say mat_A[4][4] and mat_B[4][4], this code works if i use only one process however if i use two or four process it doesnt give me right result. ah the rows or cols need to be divisible by # of process. do you guys see where the problem is? thank you. ### /r/compsci #### Things I Should Know Hey, guys. I have recently changed my major to CS and will be taking courses for it next semester. I am a little concerned about what I should come to class next semester knowing. Could you take some time out to list a few things I should know before going back to school next semester. Just to make things a little clearer, I am currently a first semester freshman, I am pretty good at math(I took Calculus 1 in high school, and I am taking Calculus 2 next semester), and I know a few programming languages. I know CS isn't completely about programming, but I thought it might help to know that. Any information would be greatly appreciated. submitted by ragnarok438 [link] [1 comment] ### StackOverflow #### Why doesn't HashMap.merged work on Map Why doesn't HashMap.merged work on Map? For example like this:  val resultMap = collection.immutable.HashMap[String, Int]() val justMap = collection.immutable.Map[String, Int]() resultMap.merged(justMap)  #### Haskell generics with unique ids I've been using Uniplate for a while, but I find the lack ability to identify a node to make things more difficult for me. Is there a generics implementation that allows absolute ids? Here is an example of some APIs I would expect such an implementation to have: universe :: (Data on) => on -> [(Id,on)] -- returns all sub nodes, with ids children :: (Data on) => on -> [(Id,on)] -- returns all direct children, with ids transformAbove :: (Data on) => (on -> on) -> Id -> on -> on -- applies a transformation to all nodes which are ancestors of the node with the given id  ### Lobsters #### Write The Docs - A series of events for technical writers, documentarians, and all those who write the docs. #### Sexy prime - Wikipedia #### AnalyzePDF - Bringing the Dirt Up to the Surface #### Stop Saying Bitcoin Transactions Aren’t Reversible ### /r/netsec #### 2 million Facebook, Gmail and Twitter passwords stolen in massive hack ### StackOverflow #### Chaining value class calls Is it possible to chain values classes without them actually being instantiated? For example: import scala.swing._ object SwingPlus { implicit class RichComponent(val self: Component) extends AnyVal { def clientProps: ClientProperties = new ClientProperties(self) } implicit class ClientProperties(val self: Component) extends AnyVal { def += (kv: (String, Any)): Component = { self.peer.putClientProperty(kv._1, kv._2) self } } }  Use case: import SwingPlus._ val b = Button("Foo") { println("bar") } b.clientProps += "JButton.buttonType" -> "textured" val f = new Frame { contents = but; pack(); open() }  In the call b.clientProps +=, does ClientProperties get instantiated, or are these just pure static (optimised) method calls? ### /r/netsec #### NSA tracking cellphone locations worldwide, Snowden documents show ### /r/freebsd #### FreeBSD Kitten sysadmin asleep on the job #newnix ### Planet Theory #### The Morse theory of \v{C}ech and Delaunay filtrations Authors: Ulrich Bauer, Herbert Edelsbrunner Download: PDF Abstract: Given a finite set of points in$\mathbb R^n$and a positive radius, we consider the \v{C}ech, Delaunay-\v{C}ech, alpha, and wrap complexes as examples of a generalized discrete Morse theory. We prove that the latter three are simple-homotopy equivalent, and the same is true for their weighted versions. Our results have applications in high-dimensional data analysis and in the reconstruction of shapes from sampled data. #### Study on Delaunay tessellations of 1-irregular cuboids for 3D mixed element meshes Authors: David Contreras, Nancy Hitschfeld-Kahler Download: PDF Abstract: Mixed elements meshes based on the modified octree approach contain several co-spherical point configurations. While generating Delaunay tessellations to be used together with the finite volume method, it is not necessary to partition them into tetrahedra; co-spherical elements can be used as final elements. This paper presents a study of all co-spherical elements that appear while tessellating a 1-irregular cuboid (cuboid with at most one Steiner point on its edges) with different aspect ratio. Steiner points can be located at any position between the edge endpoints. When Steiner points are located at edge midpoints, 24 co-spherical elements appear while tessellating 1-irregular cubes. By inserting internal faces and edges to these new elements, this number is reduced to 13. When 1-irregular cuboids with aspect ratio equal to$\sqrt{2}$are tessellated, 10 co-spherical elements are required. If 1-irregular cuboids have aspect ratio between 1 and$\sqrt{2}$, all the tessellations are adequate for the finite volume method. When Steiner points are located at any position, the study was done for a specific Steiner point distribution on a cube. 38 co-spherical elements were required to tessellate all the generated 1-irregular cubes. Statistics about the impact of each new element in the tessellations of 1-irregular cuboids are also included. This study was done by developing an algorithm that construct Delaunay tessellations by starting from a Delaunay tetrahedral mesh built by Qhull. #### Efficient Multi-Robot Motion Planning for Unlabeled Discs in Simple Polygons Authors: Aviv Adler, Mark de Berg, Dan Halperin, Kiril Solovey Download: PDF Abstract: We consider the following motion-planning problem: we are given$m$unit discs in a simple polygon with$n$vertices, each at their own start position, and we want to move the discs to a given set of$m$target positions. Contrary to the standard (labeled) version of the problem, each disc is allowed to be moved to any target position, as long as in the end every target position is occupied. We show that this unlabeled version of the problem can be solved in$O(mn+m^2)$time, assuming that the start and target positions are at least some minimal distance from each other. This is in sharp contrast to the standard (labeled) and more general multi-robot motion planning problem for discs moving in a simple polygon, which is known to be strongly NP-hard. #### Detecting intersections between convex polyhedra Authors: Luis Barba, Stefan Langerman Download: PDF Abstract: For a polyhedron$P$in$\mathbb{R}^d$, denote by$|P|$its combinatorial complexity, i.e., the number of faces, edges and vertices of the polyhedra. In this paper, we revisit the classic problem of preprocessing polyhedra independently so that given two preprocessed polyhedra$P$and$Q$in$\mathbb{R}^d$, each translated and rotated, their intersection can be tested rapidly. For$d=3$we show how to perform such a test in$O(\log |P| + \log |Q|)$time after linear preprocessing time and space. This improves upon the last best known query time of$O(\log|P| \log|Q|)$by Dobkin and Kirkpatrick (1990). We then generalize our method to any constant dimension$d$, achieving the same$O(\log |P| + \log |Q|)$query time using a representation of size$O(|P|^{\lfloor d^2/4\rfloor + \varepsilon})$for any$\varepsilon>0$arbitrarily small. This answers an even older question posed by Dobkin and Kirkpatrick 30 years ago. In addition, we provide an alternative$O(\log |P| + \log |Q|)$algorithm to test the intersection of two convex polygons$P$and$Q$in the plane. #### Bandits and Experts in Metric Spaces Authors: Robert Kleinberg, Aleksandrs Slivkins, Eli Upfal Download: PDF Abstract: In a multi-armed bandit problem, an online algorithm chooses from a set of strategies in a sequence of trials so as to maximize the total payoff of the chosen strategies. While the performance of bandit algorithms with a small finite strategy set is quite well understood, bandit problems with large strategy sets are still a topic of very active investigation, motivated by practical applications such as online auctions and web advertisement. The goal of such research is to identify broad and natural classes of strategy sets and payoff functions which enable the design of efficient solutions. In this work we study a very general setting for the multi-armed bandit problem in which the strategies form a metric space, and the payoff function satisfies a Lipschitz condition with respect to the metric. We refer to this problem as the "Lipschitz MAB problem". We present a solution for the multi-armed bandit problem in this setting. That is, for every metric space we define an isometry invariant which bounds from below the performance of Lipschitz MAB algorithms for this metric space, and we present an algorithm which comes arbitrarily close to meeting this bound. Furthermore, our technique gives even better results for benign payoff functions. We also address the full-feedback ("best expert") version of the problem, where after every round the payoffs from all arms are revealed. #### Design and Analysis of an Estimation of Distribution Approximation Algorithm for Single Machine Scheduling in Uncertain Environments Authors: Boris Mitavskiy, Jun He Download: PDF Abstract: In the current work we introduce a novel estimation of distribution algorithm to tackle a hard combinatorial optimization problem, namely the single-machine scheduling problem, with uncertain delivery times. The majority of the existing research coping with optimization problems in uncertain environment aims at finding a single sufficiently robust solution so that random noise and unpredictable circumstances would have the least possible detrimental effect on the quality of the solution. The measures of robustness are usually based on various kinds of empirically designed averaging techniques. In contrast to the previous work, our algorithm aims at finding a collection of robust schedules that allow for a more informative decision making. The notion of robustness is measured quantitatively in terms of the classical mathematical notion of a norm on a vector space. We provide a theoretical insight into the relationship between the properties of the probability distribution over the uncertain delivery times and the robustness quality of the schedules produced by the algorithm after a polynomial runtime in terms of approximation ratios. #### Faster and Sample Near-Optimal Algorithms for Proper Learning Mixtures of Gaussians Authors: Constantinos Daskalakis, Gautam Kamath Download: PDF Abstract: We provide an algorithm for properly learning mixtures of two single-dimensional Gaussians without any separability assumptions. Given$\tilde{O}(1/\varepsilon^2)$samples from an unknown mixture, our algorithm outputs a mixture that is$\varepsilon$-close in total variation distance, in time$\tilde{O}(1/\varepsilon^{7.01})$. Our sample complexity is optimal up to logarithmic factors, and significantly improves upon both Kalai et al, whose algorithm has a prohibitive dependence on$1/\varepsilon$, and Feldman et al., whose algorithm requires bounds on the mixture parameters and depends pseudo-polynomially in these parameters. #### A Note on the Quantum Collision Problem for Random Functions Authors: Mark Zhandry Download: PDF Abstract: The results showing a quantum query complexity of$\Theta(N^{1/3})$for the collision problem do not apply to random functions. The issues are two-fold. First, the$\Omega(N^{1/3})$lower bound only applies when the range is no larger than the domain, which precludes many of the cryptographically interesting applications. Second, most of the results in the literature only apply to$r$-to-1 functions, which are quite different from random functions. Understanding the collision problem for random functions is of great importance to cryptography, and we seek to fill the gaps of knowledge for this problem. To that end, we prove that, as expected, a quantum query complexity of$\Theta(N^{1/3})$holds for all interesting domain and range sizes. Our proofs are simple, and combine existing techniques with several novel tricks to obtain the desired results. ### StackOverflow #### This is not a monad, right? I've seen two sources that say the following is an identity monad. But I don't think that's true because it fails the Third Monad Law. (defn result [v] (fn [] v)) (defn bind [mv f] (f (mv))  Fails the Third Monad Law (bind (bind (result 3) identity) identity)  The inner bind call returns 3, which is not a monadic value, so Java returns the error of trying to call the function (3). Am I missing something? ### TheoryOverflow #### Reference for Turing to many-one reductions I am looking for a reference on reducing' Turing reductions to many-one reductions. I have in mind a statement of the following form (similar enough statements would also satisfy me): Theorem. If$\mathsf{A}\leq_T^f \mathsf{B}$, then$\mathsf A\leq_m^{2^f}\mathsf B^{tt}$. where "$\leq_T^f$" and "$\leq_m^f$" denote respectively Turing and many-one reductions in deterministic time$f(n)$, and "$\mathsf{B}^{tt}$" denotes a truth table' variant of the language$\mathsf{B}$, which evaluates a Boolean combination of checks "$x\in\mathsf{B}$". Proof idea for the statement: Simulate the$f(n)$-time bounded oracle Turing machine used in the Turing reduction: it's easy enough to obtain a nondeterministic Turing transducer also in time$f(n)$that guesses the answers of the$\mathsf B$oracle and writes a conjunction of checks "$x\in\mathsf B$" or "$x\not\in\mathsf{B}$" on the output, to be evaluated by an$\mathsf B^{tt}$machine. This transducer can be determinized by exploring both outcomes of the oracle calls, and handling them through disjunctions in the output; it now works in time$2^{f(n)}$. Oddly enough, I cannot seem to find any related result in complexity textbooks. Edit: renamed "$A\mathsf B$" into "$\mathsf B^{tt}$" to emphasize the relation with truth tables, as pointed out by @MarkusBläser. ### /r/emacs #### Is it possible to use global-set-key to set command-history for python interpreter? This question is from using emacs as a Python IDE. I've started using a macbook pro to do a lot of coding lately (moving from Linux) and I was used to using CTRL+UP-ARROW to navigate through recent commands, but on OSX 10.8.5 that opens Mission Control and allows you to toggle applications or spaces (which I kind of like). I figured I would remap M-UP and M-Down to cycle through command history (these are not currently bound to anything) but could not find information on how to do that (perhaps because it's specific to the interpreter?). Does anyone know how to do this? Thanks for your help. submitted by erewok [link] [4 comments] ### StackOverflow #### How to set host in Scalatra? How do I set the host in Scalatra to 0.0.0.0, so that it starts on that host from > container:start? The docs tells one how to set the port; but not how to set the host. ### arXiv Cryptography and Security #### Periodic Structure of the Exponential Pseudorandom Number Generator. (arXiv:1312.1311v1 [math.NT]) We investigate the periodic structure of the exponential pseudorandom number generator obtained from the map$x\mapsto g^x\pmod p$that acts on the set$\{1, \ldots, p-1\}$. #### Algebraic Principles for Rely-Guarantee Style Concurrency Verification Tools. (arXiv:1312.1225v1 [cs.DC]) We provide simple equational principles for deriving rely-guarantee-style inference rules and refinement laws based on idempotent semirings. We link the algebraic layer with concrete models of programs based on languages and execution traces. We have implemented the approach in Isabelle/HOL as a lightweight concurrency verification tool that supports reasoning about the control and data flow of concurrent programs with shared variables at different levels of abstraction. This is illustrated on two simple verification examples. #### Scalability of the plasma physics code GEM. (arXiv:1312.1187v1 [cs.DC]) We discuss a detailed weak scaling analysis of GEM, a 3D MPI-parallelised gyrofluid code used in theoretical plasma physics at the Max Planck Institute of Plasma Physics, IPP at Garching b. M\"unchen, Germany. Within a PRACE Preparatory Access Project various versions of the code have been analysed on the HPC systems SuperMUC at LRZ and JUQUEEN at J\"ulich Supercomputing Centre (JSC) to improve the parallel scalability of the application. The diagnostic tool Scalasca has been used to filter out suboptimal routines. The code uses the electromagnetic gyrofluid model which is a superset of magnetohydrodynamic and drift-Alfv\'en microturbulance and also includes several relevant kinetic processes. GEM can be used with different geometries depending on the targeted use case, and has been proven to show good scalability when the computational domain is distributed amongst two dimensions. Such a distribution allows grids with sufficient size to describe small scale tokamak devices. In order to enable simulation of very large tokamaks (such as the next generation nuclear fusion device ITER in Cadarache, France) the third dimension has been parallelised and weak scaling has been achieved for significantly larger grids. #### Circular-arc hypergraphs: Rigidity via Connectedness. (arXiv:1312.1172v1 [cs.DM]) A circular-arc hypergraph$H$is a hypergraph admitting an arc ordering, that is, a circular ordering of the vertex set$V(H)$such that every hyperedge is an arc of consecutive vertices. An arc ordering is tight if, for any two hyperedges$A$and$B$such that$A$is a nonempty subset of$B$and$B$is not equal to$V(H)$, the corresponding arcs share a common endpoint. We give sufficient conditions for$H$to have, up to reversing, a unique arc ordering and a unique tight arc ordering. These conditions are stated in terms of connectedness properties of$H$. It is known that$G$is a proper circular-arc graph exactly when its closed neighborhood hypergraph$N[G]$admits a tight arc ordering. We explore connectedness properties of$N[G]$and prove that, if$G$is a connected, twin-free, proper circular-arc graph with non-bipartite complement, then$N[G]$has, up to reversing, a unique arc ordering. If the complement of$G$is bipartite and connected, then$N[G]$has, up to reversing, two tight arc orderings. As a corollary, we notice that in both of the two cases$G$has an essentially unique intersection representation. The last result also follows from the work by Deng, Hell, and Huang based on a theory of local tournaments. #### Yao's millionaires' problem and decoy-based public key encryption by classical physics. (arXiv:1312.1160v1 [cs.CR]) We use various laws of classical physics to offer several solutions of Yao's millionaires' problem without using any one-way functions. We also describe several informationally secure public key encryption protocols, i.e., protocols secure against passive computationally unbounded adversary. This introduces a new paradigm of decoy-based cryptography, as opposed to "traditional" complexity-based cryptography. In particular, our protocols do not employ any one-way functions. #### A Game Semantics for Generic Polymorphism. (arXiv:1312.1120v1 [cs.LO]) Genericity is the idea that the same program can work at many different data types. Longo, Milstead and Soloviev proposed to capture the inability of generic programs to probe the structure of their instances by the following equational principle: if two generic programs, viewed as terms of type$\forall X. \, A[X]$, are equal at any given instance$A[T]$, then they are equal at all instances. They proved that this rule is admissible in a certain extension of System F, but finding a semantically motivated model satisfying this principle remained an open problem. In the present paper, we construct a categorical model of polymorphism, based on game semantics, which contains a large collection of generic types. This model builds on two novel constructions: -- A direct interpretation of variable types as games, with a natural notion of substitution of games. This allows moves in games A[T] to be decomposed into the generic part from A, and the part pertaining to the instance T. This leads to a simple and natural notion of generic strategy. -- A "relative polymorphic product" which expresses quantification over the type variable X in the variable type A with respect to a "universe'" which is explicitly given as an additional parameter B. We then solve a recursive equation involving this relative product to obtain a universe in a suitably "absolute" sense. Full Completeness for ML types (universal closures of quantifier-free types) is proved for this model. #### Interaction Graphs: Exponentials. (arXiv:1312.1094v1 [cs.LO]) In two previous papers, we exposed a combinatorial approach to the program of Geometry of Interaction (GoI), a program initiated by Jean-Yves Girard which aims at giving a semantics of proofs that accounts for the dynamics of cut-elimination. The strength of our approach lies in the fact that we interpret proofs by simpler structures - graphs - than Girard's constructions, while generalizing the latter since they can be recovered as special cases of our setting. This third paper tackles the complex issue of defining exponential connectives in this framework. In order to succeed in this, we consider a generalization of graphs named graphings, which is in some way a geometric realization of a graph. We explain how we can then define a GoI for Elementary Linear Logic (ELL), a sub-system of linear logic where representable functions are exactly the functions computable in elementary time. This construction is moreover parametrized by the choice of a map, giving rise to a whole family of models. #### Explicit Convergence Rate of a Distributed Alternating Direction Method of Multipliers. (arXiv:1312.1085v1 [cs.DC]) Consider a set of$N$agents seeking to solve distributively the minimization problem$\inf_{x} \sum_{n = 1}^N f_n(x)$where the convex functions$f_n$are local to the agents. The popular Alternating Direction Method of Multipliers has the potential to handle distributed optimization problems of this kind. In this paper, the rate of convergence of this method is considered. It is assumed that the infimum of the problem is reached at a point$x_\star$, the functions$f_n$are twice differentiable at this point and$\sum \nabla^2 f_n(x_\star) > 0$in the positive definite ordering of symmetric matrices. With these assumptions, it is shown that the convergence to the consensus$x_\star$is linear and the exact rate is provided. Application examples where this rate can be optimized with respect to the ADMM free parameter$\rho$are also given. #### A Necessary and Sufficient Condition for the Existence of Potential Functions for Heterogeneous Routing Games. (arXiv:1312.1075v1 [cs.GT]) Studies in routing games mostly assume that vehicles are of the same type and, therefore, experience the same latency or cost when traveling along a specific edge of the transportation network. In this article, in contrast, we propose a heterogeneous routing game in which vehicles might belong to more than one type. The type determines the cost of traveling along an edge as a function of the flow of various types of vehicles over that edge. We prove the existence of a Nash equilibrium in this heterogeneous routing game. We also present necessary and sufficient conditions for the existence of a potential function. Under these conditions, we can pose the problem of finding a Nash equilibrium for the heterogeneous routing game as an optimization problem. We characterize a set of tolls that guarantee the existence of a potential function. We present an upper bound for the price of anarchy (i.e., the worst-case ratio of the social cost calculated for a Nash equilibrium over the social cost for a socially optimal flow) for the case in which only two types of players are participating in a heterogeneous routing game with affine edge cost functions. A heterogeneous routing game with vehicle platooning incentives is used as an example throughout the article to clarify the concepts and to validate the results. #### Checking Temporal Properties of Presburger Counter Systems using Reachability Analysis. (arXiv:1312.1070v1 [cs.LO]) Counter systems are a well-known and powerful modeling notation for specifying infinite-state systems. In this paper we target the problem of checking temporal properties of counter systems. We first focus on checking liveness properties only, and propose two semi decision techniques for these properties. Both these techniques return a formula that encodes the set of reachable states of a given system that satisfy a given liveness property. A novel aspect of our techniques is that they use reachability analysis techniques, which are well studied in the literature, as black boxes, and are hence able to compute precise answers on a much wider class of systems than previous approaches for the same problem. Secondly, they compute their results by iterative expansion or contraction, and hence permit an approximate solution to be obtained at any point. We state the formal properties of our techniques, and also provide experimental results using standard benchmarks to show the usefulness of our approaches. Finally, we provide a technique for checking arbitrary CTL temporal properties, which use the liveness property checking techniques mentioned above as black boxes. #### On Theoretical Analysis of Distributed Stochastic Dual Coordinate Ascent. (arXiv:1312.1031v1 [cs.DC]) In \citep{Yangnips13}, the author presented distributed stochastic dual coordinate ascent (DisDCA) algorithms for solving large-scale regularized loss minimization. Extraordinary performances have been observed and reported for the well-motivated updates, as referred to the practical updates, compared to the naive updates. However, no serious analysis has been provided to understand the updates and therefore the convergence rates. In the paper, we bridge the gap by providing a theoretical analysis of the convergence rates of the practical DisDCA algorithm. Our analysis helped by empirical studies has shown that it could yield an exponential speed-up in the convergence by increasing the number of dual updates at each iteration. This result justifies the superior performances of the practical DisDCA as compared to the naive variant. As a byproduct, our analysis also reveals the convergence behavior of the one-communication DisDCA. #### The Truth Behind the Myth of the Folk Theorem. (arXiv:1312.1017v1 [cs.GT]) We study the problem of computing an$\epsilon$-Nash equilibrium in repeated games. Earlier work by Borgs et al.[2010] suggests that this problem is intractable. We show that if we make a slight change to their model---modeling the players as polynomial-time Turing machines that maintain state (rather than stateless polynomial-time Turing machines)---and make some standard cryptographic hardness assumptions (the existence of public-key encryption), the problem can actually be solved in polynomial time. #### Impact of Report Message Scheduling (RMS) in 1G/10G EPON and GPON (Extended Version). (arXiv:1312.0994v1 [cs.NI]) A wide array of dynamic bandwidth allocation (DBA) mechanisms have recently been proposed for improving bandwidth utilization and reducing idle times and packets delays in passive optical networks (PONs). The DBA evaluation studies commonly assumed that the report message for communicating the bandwidth demands of the distributed optical network units (ONUs) to the central optical line terminal (OLT) is scheduled for the end of an ONU's upstream transmission, after the ONU's payload data transmissions. In this article, we conduct a detailed investigation of the impact of the report message scheduling (RMS), either at the beginning (i.e., before the pay load data) or the end of an ONU upstream transmission on PON performance. We analytically characterize the reduction in channel idle time with reporting at the beginning of an upstream transmission compared to reporting at the end. Our extensive simulation experiments consider both the Ethernet Passive Optical Networking (EPON) standard and the Gigabit PON (GPON) standard. We find that for DBAs with offline sizing and scheduling of ONU upstream transmission grants at the end of a polling cycle, which processes requests from all ONUs, reporting at the beginning gives substantial reductions of mean packet delay at high loads. For high-performing DBAs with online grant sizing and scheduling, which immediately processes individual ONU requests, or interleaving of ONUs groups, both reporting at the beginning or end give essentially the same average packet delays. #### TRAIL: Topology Authentication in RPL. (arXiv:1312.0984v1 [cs.NI]) The IPv6 Routing Protocol for Low-Power and Lossy Networks was recently introduced as the new routing standard for the Internet of Things. Although RPL defines basic security modes, it remains vulnerable to topological attacks which facilitate blackholing, interception, and resource exhaustion. We are concerned with analyzing the corresponding threats and protecting future RPL deployments from such attacks. Our contributions are twofold. First, we analyze the state of the art, in particular the protective scheme VeRA and present two new rank order attacks as well as extensions to mitigate them. Second, we derive and evaluate TRAIL, a generic scheme for topology authentication in RPL. TRAIL solely relies on the basic assumptions of RPL that (1) the root node serves as a trust anchor and (2) each node interconnects to the root in a straight hierarchy. Using proper reachability tests, TRAIL scalably and reliably identifies any topological attacker without strong cryptographic efforts. #### Attacks on Fixed Apparatus Quantum Key Distribution Schemes. (arXiv:1312.0979v1 [quant-ph]) We consider quantum key distribution implementations in which the receiver's apparatus is fixed and does not depend on his choice of basis at each qubit transmission. We show that, although theoretical quantum key distribution is proven secure, such implementations are totally insecure against a strong eavesdropper that has one-time (single) access to the receiver's equipment. The attack we present here, the "fixed-apparatus attack" causes a potential risk to the usefulness of several recent implementations. #### Checkpointing algorithms and fault prediction. (arXiv:1302.3752v2 [cs.DC] UPDATED) This paper deals with the impact of fault prediction techniques on checkpointing strategies. We extend the classical first-order analysis of Young and Daly in the presence of a fault prediction system, characterized by its recall and its precision. In this framework, we provide an optimal algorithm to decide when to take predictions into account, and we derive the optimal value of the checkpointing period. These results allow to analytically assess the key parameters that impact the performance of fault predictors at very large scale. #### A space efficient streaming algorithm for triangle counting using the birthday paradox. (arXiv:1212.2264v3 [cs.DS] UPDATED) We design a space efficient algorithm that approximates the transitivity (global clustering coefficient) and total triangle count with only a single pass through a graph given as a stream of edges. Our procedure is based on the classic probabilistic result, the birthday paradox. When the transitivity is constant and there are more edges than wedges (common properties for social networks), we can prove that our algorithm requires$O(\sqrt{n})$space ($n$is the number of vertices) to provide accurate estimates. We run a detailed set of experiments on a variety of real graphs and demonstrate that the memory requirement of the algorithm is a tiny fraction of the graph. For example, even for a graph with 200 million edges, our algorithm stores just 60,000 edges to give accurate results. Being a single pass streaming algorithm, our procedure also maintains a real-time estimate of the transitivity/number of triangles of a graph, by storing a minuscule fraction of edges. #### Recognizing Interval Bigraphs by Forbidden Patterns. (arXiv:1211.2662v2 [cs.DS] UPDATED) Let H be a connected bipartite graph with n nodes and m edges. We give an O(n(m+n)) time algorithm to decide whether H is an interval bigraph. The best known algorithm has time complexity O(nm^6(m + n) \log n) and it was developed in 1997 [15]. Our approach is based on an ordering characterization of interval bigraphs introduced by Hell and Huang [11]. We transform the problem of finding the desired ordering to choosing strong components of a pair-digraph without creating conflicts. We make use of the structure of the pair-digraph as well as decomposition of bigraph H based on the special components of the pair-digraph. This way we make explicit what the difficult cases are and gain efficiency by isolating such situations. We believe our method can be used to find a desired ordering for other classes of graphs and digraphs having ordering characterization. #### Algorithms and Hardness for Robust Subspace Recovery. (arXiv:1211.1041v3 [cs.CC] UPDATED) We consider a fundamental problem in unsupervised learning called \emph{subspace recovery}: given a collection of$m$points in$\mathbb{R}^n$, if many but not necessarily all of these points are contained in a$d$-dimensional subspace$T$can we find it? The points contained in$T$are called {\em inliers} and the remaining points are {\em outliers}. This problem has received considerable attention in computer science and in statistics. Yet efficient algorithms from computer science are not robust to {\em adversarial} outliers, and the estimators from robust statistics are hard to compute in high dimensions. Are there algorithms for subspace recovery that are both robust to outliers and efficient? We give an algorithm that finds$T$when it contains more than a$\frac{d}{n}$fraction of the points. Hence, for say$d = n/2$this estimator is both easy to compute and well-behaved when there are a constant fraction of outliers. We prove that it is Small Set Expansion hard to find$T$when the fraction of errors is any larger, thus giving evidence that our estimator is an {\em optimal} compromise between efficiency and robustness. As it turns out, this basic problem has a surprising number of connections to other areas including small set expansion, matroid theory and functional analysis that we make use of here. #### A Note on Circular Arc Online Coloring using First Fit. (arXiv:1211.0297v2 [cs.DS] UPDATED) In Raman (2007), using a column construction technique it is proved that every interval graph can be colored online with First Fit with at most$8w(G)$colors, where$w(G)$is the size of the maximum clique of$G$. Since the column construction can not be adapted to circular arc graphs we give a different proof to establish an upper bound of$9w(G)$for online coloring a circular arc graph$G$with the First Fit algorithm. ### StackOverflow #### Data structure for non overlapping ranges of integers? I remember learning a data structure that stored a set of integers as ranges in a tree, but it's been 10 years and I can't remember the name of the data structure, and I'm a bit fuzzy on the details. If it helps, it's a functional data structure that was taught at CMU, I believe in 15-212 (Principles of Programming) in 2002. Basically, I want to store a set of integers, most of which are consecutive. I want to be able to query for set membership efficiently, add a range of integers efficiently, and remove a range of integers efficiently. In particular, I don't care to preserve what the original ranges are. It's better if adjacent ranges are coalesced into a single larger range. A naive implementation would be to simply use a generic set data structure such as a HashSet or TreeSet, and add all integers in a range when adding a range, or remove all integers in a range when removing a range. But of course, that would waste a lot of memory in addition to making add and remove slow. I'm thinking of a purely functional data structure, but for my current use I don't need it to be. IIRC, lookup, insertion, and deletion were all O(log N), where N was the number of ranges in the set. So, can you tell me the name of the data structure I'm trying to remember, or a suitable alternative? ### CompsciOverflow #### LR(1) constructing DFA Consider the following grammar, where$ [ \ ] \ | \ , \ $are terminal symbols$ \begin{eqnarray} L \rightarrow [ \ ] \qquad(1) \\ L \rightarrow [ \ H\ T\ ] \qquad(2) \\ H \rightarrow L \qquad (3) \\ H \rightarrow L\ ,\ H \qquad (4) \\ T \rightarrow |\ L \qquad(5) \\ T \rightarrow \epsilon \qquad (6) \end{eqnarray} $I went over the step to create the DFA for that grammar, and then building parsing table. However, after creating those things, I cannot parse the string$ [\ [\ ]\ |\ [\ [\ ]\ ,\ [\ [\ ]\ ]\ |\ [\ [\ ]\ ]\ ]\ ] $Here is the DFA: Note: In state 11, the only lookahead is$


I think my problem is that I determined wrong lookahead for some DFA states. But I still don't know what's wrong here.

Can anyone give my some idea?

### Fefe

#### Die EU hat Kartellstrafen gegen Großbanken verhängt. ...

Die EU hat Kartellstrafen gegen Großbanken verhängt. Insgesamt werden jetzt 1.7 Milliarden Euro Strafe fällig. Es geht diesmal um Zinsraten und Derivate darauf. Die größten Strafen wären eigentlich für UBS und Barclays fällig gewesen, aber die haben sich sofort umgedreht und sich als Kronzeugen gegen ihre Kumpels zur Verfügung gestellt. Als Gegenleistung müssen sie jetzt gar nichts zahlen. Die größte Strafe muss jetzt die Deutsche Bank zahlen, in Höhe von 725 Millionen Euro.

### CompsciOverflow

#### Boolean Simplification, need help simplifying

(!A.!C)+(!A.D)+(!A.B)+(!B.!C)

can simplify down to

(!A.D)+(!A.B)+(!B.!C)

But i don't know what law i'm supposed to use, the truth table for both these expressions are the same. What law(s) am i supposed to use?

### /r/compsci

#### Is majoring in CS right for me?

I'm currently a junior at my high school at the moment and I've been thinking alot about what I'm going to have to major in, in college. At first, I had wanted to go into a Mechanical Engineering field but I later found out that I'm more of a computer guy and less of those Mechanical stuff, but I do seem to have a little interest in it. I aswell have the same interest in CS. What should I ask myself before majoring in this field?

submitted by xPurify

### Planet Theory

#### Algorithmic Growth (Class Size)

Pre-term planning numbers are in for Harvard, and it looks like the undergrad Algorithms and Data Structures class has about 175 people planning to take the course.  That's a huge jump again over the last few years (where it's jumped from the 50s to well over 100).  I imagine the growth is spurred by our ever-increasing enrollment numbers in the intro classes, as well as the fact that it's being taken over by a younger, dynamic, new faculty member.  (Go Jelani Nelson.  I can't help but think some number of students were waiting for me to go on sabbatical...)

These numbers are usually within plus-minus 10% of the final, though there's higher variance when new instructors take over.  If 175 became the steady state class size, it would mean a bit over 10% of the students at Harvard would take Algorithms at some point.  I don't think I ever expected that when I started.

If we can get the people resources, at some point we'll probably want to start breaking this up.  One direction is to make an "honors" class that would cover more/harder material more rapidly.  (We're thinking of making this an "honors theory" course, that would cover both complexity and algorithms -- 2 classes packed into 1.)  The Math department here has done this very successfully, separating out the future Putnam winners from other students early on.  A benefit is it leaves the remaining students happier as well, as the curve-breakers remove themselves from the regular class.  Another possibility is to do an algorithms style class designed for non-majors, that would bring in more people not currently taking algorithms as well as some of those in the current course.  There are "topics" classes like this -- the Easley/Kleinberg book Networks, Crowds, and Markets: Reasoning About a Highly Connected World is algorithmic and seems to allow an instructor to choose a desired mathematical level from a broad range -- but I don't really know of examples of something more like a standard algorithms/data structures courses designed for a broader audience than for CS majors.  I'd be interested in pointers to and anecdotes about experiences in such classes if they exist.

### /r/compsci

#### Trying to track an abusive IP on twitter. Need help masking a URL to make it not suspicious or other. PLEASE

A friend of mine went through something very traumatic and someone created a twitter account and is harassing her to the point where she is well... to say the least... very very upset. And I want to help track this person down so I can perhaps help her figure out who is doing it so she can resolve the issue or press charges if need be.

I was going to use blasze IP logger to trick the person into clicking on a link to capture their IP, then sending messages to this girls friends and get them to capture their IP's, then comparing the results to find a match.

The problem is when I send them to a link, it sends them to blasze.t k/(some jumble), captures their IP, then redirects them instantly to the site I am sending them to. But sending them to a site named blasze.t k/(somejumble) looks suspicious so I used goo.g l to mask the URL, but I am worried this is STILL too suspicious. I'm not amazing with computers I was wondering if there is a way to make the link even less suspicious than this, or some way to track this person easier? Any advice would be wonderful.

Edit: Why are people downvoting, my friend is near ready to do something very extreme and it's a very stressful situation. If you think I'm doing something stupid please tell me and offer me advice.

submitted by kyleswimmer87

### StackOverflow

#### Clojure indenter that works

I am looking for an editor/command line program that properly (IMHO) indents Clojure code, aligning the third and subsequent elements with the second element:

(let [foo (fun arg1
arg2)

bar (doto (Baz.)
(.setProp abc123))

[v1 v2] (fun2)

[v3 v4] (fun3)]
(xyzzy 42))


I have tried La Clojure for IntelliJ with the "Align Clojure forms by the second element" (does not align by the second element), VimClojure, emacs (messed up the indentation of the [v3 v4] part), Enclojure in Netbeans (does not align by the second element), and Counterclockwise in Eclipse (does not align by the second element). In each case, I am using the latest version of the software.

I can run a beautifier outside of the editor (on the command line) if necessary.

Here is the output of Aquamacs. Note the "cascading" (make-day-panel) calls:

(defn make-mainframe [build-number project-version]
"Create the mainframe.
@param build-number the current build number
@param project-version the Maven project version
@returns the reified JFrame"
(let [[^JToggleButton day-sheets-button
^JToggleButton summaries-button]
(let [day-sheets-button
(doto (JToggleButton. "Day Sheets")
(.setToolTipText
"Switch to the Day Sheets view"))

summaries-button
(doto (JToggleButton. "Summaries")
(.setToolTipText "Switch to the Summaries view"))]
(doto (ButtonGroup.)
(.setSelected day-sheets-button true)
[day-sheets-button summaries-button])

[^JPanel monday-panel
monday-button
monday-vacation-check
monday-arrive-text-field
monday-depart-text-field] (make-day-panel "Monday")

[^JPanel tuesday-panel
tuesday-button
tuesday-vacation-check
tuesday-arrive-text-field
tuesday-depart-text-field] (make-day-panel "Tuesday")

[^JPanel wednesday-panel
wednesday-button
wednesday-vacation-check
wednesday-arrive-text-field
wednesday-depart-text-field] (make-day-panel "Wednesday")

[^JPanel thursday-panel
thursday-button
thursday-vacation-check
thursday-arrive-text-field
thursday-depart-text-field] (make-day-panel "Thursday")

[^JPanel friday-panel
friday-button
friday-vacation-check
friday-arrive-text-field
friday-depart-text-field] (make-day-panel "Friday")

[^JPanel saturday-panel
saturday-button
saturday-vacation-check
saturday-arrive-text-field
saturday-depart-text-field] (make-day-panel "Saturday")

[^JPanel sunday-panel
sunday-button
sunday-vacation-check
sunday-arrive-text-field
sunday-depart-text-field] (make-day-panel "Sunday")

#_ [week-selector-panel week-selector-combobox week-selector-today-button]
#_ (make-week-selector)

north-panel
(let [^JPanel panel (JPanel.)]
(doto panel
(.setLayout (BoxLayout. panel BoxLayout/X_AXIS))

^JToggleButton mail-button
(make-button "Mail Report"
(make-icon "email-32x32-plain.png")
"Generate and send a weekly report for the currently selected week")

^JToggleButton report-button
(make-button "Report"
(make-icon "history2-32x32-plain.png")
"Generate a weekly report for the currently selected week")

day-sheet-panel
(let [panel (JPanel. (BorderLayout.))

north-panel (grid-bag-container (JPanel. (GridBagLayout.))
{:insets (Insets. 2 2 2 2)}
[[(doto (JPanel. (GridLayout. 1 0))
[(Box/createHorizontalGlue)
:weightx 1.0 :fill :HORIZONTAL ]
[(doto (JPanel. (GridLayout. 1 0))
(doto panel
(.setBorder (BorderFactory/createRaisedBevelBorder))))

summaries-panel (let [panel (JPanel.)]
(doto panel))

card-panel (let [panel (JPanel. (CardLayout.))]
(doto panel

main-panel (doto (JPanel. (BorderLayout.))
(.setBorder
(BorderFactory/createEmptyBorder default-border default-border
default-border default-border))

frame (doto (JFrame. (str "SpareTime (version " project-version ", build "
build-number ")"))
(.setDefaultCloseOperation JFrame/EXIT_ON_CLOSE)
(reify mainview-methods
(frame-center [_] (.setLocationRelativeTo frame nil))
(frame-get-location [_] (.getLocation frame))
(frame-get-size [_] (.getSize frame))
(frame-set-location [_ x y] (.setLocation frame x y))
(frame-set-size [_ width height] (.setSize frame (Dimension. width height)))
(frame-set-visible [_ visible] (.setVisible frame visible)))))


#### How to print backtrace in a freebsd machine?

I have written a code to print backtrace in a linux machine using a backtrace function call specified in the "execinfo.h" header file. I need to get this code working on a freebsd machine as well. I am getting a compiler error for "execinfo.h" in Freebsd machines. I am thinking of using conditional macros to solve the problem, but I am not able to find a proper documentation for printing backtrace in Freebsd machines.

#### Play 2.1 Scala SQLException Connection Timed out waiting for a free available connection

I have been working on this issue for quite a while now and I cannot find a solution...

A web app built with play framework 2.2.1 using h2 db (for dev) and a simple Model package. I am trying to implement a REST JSON endpoint and the code works... but only once per server instance.

def createOtherModel() = Action(parse.json) {
request =>
request.body \ "name" match {
case _: JsUndefined => BadRequest(Json.obj("error" -> true,
"message" -> "Could not match name =(")).as("application/json")
case name: JsValue =>
request.body \ "value" match {
case _: JsUndefined => BadRequest(Json.obj("error" -> true,
"message" -> "Could not match value =(")).as("application/json")
case value: JsValue =>
// this breaks the secod time
val session = ThinkingSession.dummy
val json = Json.obj(
"content" -> value,
"thinkingSession" -> session.id,
)
)
Ok(Json.obj("content" -> json)).as("application/json")
}
} else {
"message" -> "Name was not content =(")).as("application/json")
}
}
}


so basically I read the JSON, echo the "value" value, create a model obj and send it's id.

the ThinkingSession.dummy function does this:

def all(): List[ThinkingSession] = {

// Tried explicitly closing connection, no difference
//val conn = DB.getConnection()
//try {
//  DB.withConnection { implicit conn =>
//    SQL("select * from thinking_session").as(ThinkingSession.DBParser *)
//  }
//} finally {
//  conn.close()
//}
DB.withConnection { implicit conn =>
SQL("select * from thinking_session").as(ThinkingSession.DBParser *)
}
}

def dummy: ThinkingSession = {
}


So this should do a SELECT * FROM thinking_session, create a model obj list from the result and return the first out of the list.

This works fine the first time after server start but the second time I get a

play.api.Application$$anon1: Execution exception[[SQLException: Timed out waiting for a free available connection.]] at play.api.Applicationclass.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1] at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1] at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfunapplyOrElse3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1] at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfunapplyOrElse3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1] at scala.Option.map(Option.scala:145) [scala-library.jar:na] at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2.applyOrElse(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1] Caused by: java.sql.SQLException: Timed out waiting for a free available connection. at com.jolbox.bonecp.DefaultConnectionStrategy.getConnectionInternal(DefaultConnectionStrategy.java:88) ~[bonecp.jar:na] at com.jolbox.bonecp.AbstractConnectionStrategy.getConnection(AbstractConnectionStrategy.java:90) ~[bonecp.jar:na] at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:553) ~[bonecp.jar:na] at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:131) ~[bonecp.jar:na] at play.api.db.DBApi$class.getConnection(DB.scala:67) ~[play-jdbc_2.10.jar:2.2.1]
at play.api.db.BoneCPApi.getConnection(DB.scala:276) ~[play-jdbc_2.10.jar:2.2.1]


My application.conf (db section)

db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:file:database/[my_db]"
db.default.logStatements=true
db.default.idleConnectionTestPeriod=5 minutes
db.default.connectionTestStatement="SELECT 1"
db.default.maxConnectionAge=0
db.default.connectionTimeout=10000


Initially the only thing set in my config was the connection and the error occurred. I added all the other stuff while reading up on the issue on the web.

What is interesting is that when I use the h2 in memory db it works once after server start and after that it fails. when I use the h2 file system db it only works once, regardless of the server instances.

Can anyone give me some insight on this issue? Have found some stuff on bonecp problem and tried upgrading to 0.8.0-rc1 but nothing changed... I am at a loss =(

### Fefe

#### Hey, wenn die GSG 9 schon unterwegs ist, dann kann ...

Hey, wenn die GSG 9 schon unterwegs ist, dann kann sie ja auch gleich mal ein Bitcoin-Mining-Botnet hopsnehmen. Was heute alles unter Terrorismus fällt...

#### Die NSA trackt gewohnheitsmäßig den Aufenthaltsort ...

Die NSA trackt gewohnheitsmäßig den Aufenthaltsort von "at least hundreds of millions" von Mobiltelefonen. Hunderte Millionen! Das ist schon krass. Ein weiteres Mal muss sich das "ach das ist aufwendig, und so viel bringt das ja auch nicht, das wird schon niemand machen, und überhaupt, wer interessiert sich schon für mich" der Realität geschlagen geben.

#### Netzneutralitäts-Update aus den USA: Der frisch ernannte ...

Netzneutralitäts-Update aus den USA: Der frisch ernannte FCC-Boss sagt erstmal an, dass er das okay fände, wenn die ISPs von Netflix Kohle sehen wollen. Die FCC ist die zuständige Regulierungsbehörde, und hat bisher Netzneutralität hochgehalten.

#### In Mexiko ist ein VW-Transporter mit Kobalt-60 gestohlen ...

In Mexiko ist ein VW-Transporter mit Kobalt-60 gestohlen worden. Der radioaktive Sondermüll war auf dem Weg von einem Krankenhaus zu einem Lager. Kobalt wird in Krankenhäusern für Strahlentherapie eingesetzt und ist hochgefährlich, wenn es aus dem Schutzcontainer entnommen wird. Es gab da schonmal so einen Fall, der ziemlich berühmt ist.

Inzwischen berichtet die Gerüchteküche, der Truck sei ohne das Kobalt gefunden worden. Das wäre ausgesprochen gruselig, weil es darauf hindeuten würde, dass da jemand tatsächlich das strahlende Material stehlen wollte. Angesichts der furchtbaren Gewaltexzesse zwischen Polizei und Drogenbaronen in Mexiko kann man sich da diverse Gruselszenarios ausdenken.

Update: Das Zeug ist gefunden worden, allerdings außerhalb des Transporters. Der ist aber wohl in einer unbewohnten Gegend gefunden worden und daher evakuieren sie jetzt nicht.

#### Die US-Heimatschutzbehörde hat ein Verfahren wegen ...

Die US-Heimatschutzbehörde hat ein Verfahren wegen ihrer "no-fly list" am Hals. Hmm, das könnte potentiell peinlich werden. Was tun, sprach Zeus? Die lupenreinen Demokraten des DHS haben eine elegante Lösung gefunden:
On the first day of trial, the judge learned that the plaintiff's daughter, scheduled to testify, was delayed because she had been denied boarding of her flight because she was put a Department of Homeland Security no-fly list.
Die haben die eine peinliche Zeugin erstmal auf die no-fly-Liste gesetzt, damit sie nicht vor Gericht gegen sie aussagen kann! Das setzt echt Maßstäbe. Grandios!

#### Wie katastrophal wird die große Koalition? So katastrophal: ...

Wie katastrophal wird die große Koalition? So katastrophal: Die "Minderheitenschutz"-Versprechen für die Opposition stellen sich als heiße Luft heraus.
Es zeichnet sich ab, dass CDU und CSU am liebsten nur leichte Veränderungen vornehmen wollen. In der Union steigen die Sympathien für das Modell einer freiwilligen Selbstverpflichtung.
Hahahaha, freiwillige Selbstverpflichtung! Na DANN ist ja alles gut! Das ist ja geradezu ein Erfolgsmodell, funktioniert immer!1!!

### Planet Theory

#### The self-assembly of paths and squares at temperature 1

Authors: Pierre-Étienne Meunier
Abstract: We prove that the number of tile types required to build squares of size n x n, in Winfree's abstract Tile Assembly Model, when restricted to using only non-cooperative tile bindings, is at least 2n-1, which is also the best known upper bound. Non-cooperative self-assembly, also known as temperature 1, is where tiles bind to each other if they match on one or more sides, whereas in cooperative binding, some tiles can bind only if they match on multiple sides.

Our proof introduces a new programming technique for temperature 1, that disproves the very intuitive and commonly held belief that, in the same model, assembling paths between two points A and B cannot be done with less tile types than the Manhattan distance between them. Then, we prove a necessary condition for these "efficient paths" to be assembled, and show that this necessary condition cannot hold in completely filled squares.

This result proves the oldest conjecture in algorithmic self-assembly, published by Rothemund and Winfree in STOC 2000, in the case where growth starts from a corner of the square. As a corollary, we establish n as a lower bound on the tile complexity of the general case. The problem of determining the minimal number of tile types to self-assemble a shape is known to be Sigma^p_2-complete.

## December 04, 2013

### QuantOverflow

#### Securitization of a loan portfolio

I am interested in the securitazion process and am looking for useful examples to help me understand how it can be applied in practice.

Assuming I have a loan portfolio consisting of secured* and unsecured loans. The loans are further assigned a (internally assigned) risk rating. The loan payments are in a local currency.

Assuming I want to securitize this loan portfolio into tranches identified by:

• secured/unsecured
• risk rating
• currency exposure hedged* (or not)

My question is

How would I go about securitizing my existing loan portfolio into the tranches described above? What are the required steps?

Could anyone recommend an spreadsheet template online that I can use as a starting point for this exercise?

[[ Notes ]]

1. The collateral backing the secured loans fall into two categories: moveable and immovable assets

2. For the sake of simplicity/familiarity etc. assume that the currency hedging is done against the USD.

### TheoryOverflow

#### How hard is it to count the number of local optima for a problem in PLS?

For a polynomial local search problem, we know that at least one solution (local optimum) must exist. However, many more solutions could exist, how hard is it to count the number of solutions for a PLS-complete problem? I am particularly interested in the decision problem: does this PLS-complete problem's instance have two or more solutions?

Does the complexity depend on which PLS-complete problem we choose? If so then I would be particularly interested in weighted 2SAT (as defined in [SY91] and [Rou10]). I know that counting the number of satisfying solutions for 2SAT is #P-complete, but at first glance it seems like local optima of weighted 2SAT and solutions for 2SAT don't have all that much in common.

I also know that for PLS's cousin PPAD, [CS02] shows that counting the number of Nash equilibria is #P-hard. This suggests that similar PLS problems like counting the number of pure-strategy equilibria in congestion games, would also be hard.

### References

[CS02] Conitzer, V., and Sandholm, T. (2002). Complexity results about Nash equilibria. IJCAI-03. cs/0205074.

[Rou10] T. Roughgarden. (2010). Computing equilibria: A computational complexity perspective. Economic Theory, 42:193-236.

[SY91] A.A. Schaeffer and M. Yannakakis. (1991). Simple local search problems that are hard to solve. SIAM Journal on Computing, 20(1):56-87.

### StackOverflow

#### Leap Motion Java SDK, java.library.path

I'm using LeapJava.jar to write a simple JDK program to interface with the Leap. I've roughly followed this article: Developing for the Leap Motion controller in Scala and in theory it's all very simple. However, when I try to run the program it fails with a RuntimeException when it tries to connect to the Leap device:

Native code library failed to load.
java.lang.RuntimeException: Nonzero exit code: 1
at scala.sys.package$.error(package.scala:27)  I've followed the steps in the Leap documentation but since I'm using Ubuntu (and sbt) there aren't clear instructions that relate directly to this situation. I've tried various different parameters for the java.library.path argument: • pointing at the ./lib dir in the project (which contains LeapJava.jar) • using the Leap SDK's lib directory • absolute path to the Leap SDK's lib directory • the x86 dir within the LeapSDK's lib directory • symlinking the LeapJava.jar file into the x86 directory and using that I get the same problem running it in IntelliJ (set up as described in the Leap docs linked above) or from the command line. e.g. sbt -Djava.library.path=.:/<path to LeapSDK>/lib run  I've added the LeapJava.jar to the classpath in sbt (it's in the lib directory) and the project compiles fine. Any advice (particularly as I'm on Linux, which is poorly documented) would be most welcome. ### Dave Winer #### CBS coverage of the Kennedy assassination One of the things I enjoyed the most about the CBS re-run of the coverage of the 1963 assassination of President Kennedy was how little deference there was to news sources by the reporters. They asked the questions I would have had asked, as the story developed. Nothing about how people feel or what the "optics" are. In a follow-up interview with Walter Cronkite with Terry Gross in 1993, he explains how hard it was for him to put aside his own feelings to do his job, but he was satisfied that he had. This was 24 years before the famous scene in Broadcast News (1987) where William Hurt airs emotions that he doesn't have. ### /r/netsec #### Call for Presentations for AusCERT2014 is open. ### StackOverflow #### Is it possible to do the Free Monad in Clojure? There has been some outstanding work with Monads in Clojure by Konrad Hinsen, Jim Duey and Leonardo Borges. My question is - is it possible to do the Free Monad in Clojure? This is an example in Haskell from an article on Scala: data Free f r = Free (f (Free f r)) | Pure r  This is the corresponding Scala example sealed abstract class Free[S[+_], +A](implicit S: Functor[S]) { final def map[B](f: A => B): Free[S, B] = flatMap(a => Return(f(a))) final def flatMap[B](f: A => Free[S, B]): Free[S, B] = this match { case Gosub(a, g) => Gosub(a, (x: Any) => Gosub(g(x), f)) case a => Gosub(a, f) } ... }  ### /r/netsec #### Why I Donated to Help Jailbreak iOS7 & You Should Too ### StackOverflow #### Custom R function returning weird output So I'm trying to create a list of lists of data frames, basically for the purposes of passing them to multiple cores via mclapply. But that's not the part I'm having trouble with. I wrote a function to create a list of smaller data frames from one large data frame, and then applied it sequentially to break a large data frame down into a list of lists of small data frames. The problem is that when the function is called the second time (via lapply to the first list of data frames), it's adding extra small data frames to each list of data frames in the larger list. I have no idea why. I don't think it's the lapply, since when I ran the function manually on one frame from the first list it also did work. Here's the code: create_frame_list<-function(mydata,mystep,elnames){ datalim<-dim(mydata)[1] mylist<-list() init<-1 top<-mystep i<-1 repeat{ if(top < datalim){ mylist[[i]]<-assign(paste(elnames,as.character(i),sep=""),data.frame(mydata[init:top,])) } else { mylist[[i]]<-assign(paste(elnames,as.character(i),sep=""),data.frame(mydata[init:datalim,])) } if(top > datalim){break} i<-i+1 init<-top+1 top<-top+mystep } return(mylist) } test_data<-data.frame(replicate(10,sample(0:1,1000,rep=TRUE))) #Create the first list of data frames, works fine master_list<-create_frame_list(test_data,300,"bd") #check the dimensions of the data frames created, they are correct lapply(master_list,dim) #create a list of lists of data frames, doesn't work right list_list<-lapply(master_list,create_frame_list,50,"children") #check the dimensions of the data frames in the various lists. The function when called again is making extra data frames of length 2 for no reason I can see lapply(list_list,lapply,dim)  So that's it. Any help is appreciated as always. ### /r/compsci #### Is anyone aware of the method that is better than TitForTat? I'm trying to learn how to program one in python (i know, god forbid and all) and apparently there are some better, but I can't seem to find them online anywhere. submitted by FREE_ALEW [link] [5 comments] ### StackOverflow #### Calling OCaml-wrapped ZeroMQ code from signal handler I've written some OCaml bindings for CZMQ based on the guide at http://www.linux-nantes.org/~fmonnier/ocaml/ocaml-wrapping-c.php, which seem to work pretty well. For example here's zstr_send: CAMLprim value caml_zstr_send(value socket_val, value string_val) { CAMLparam2 (socket_val, string_val); void *sock = CAML_CZMQ_zsocket_val(socket_val); char *string = String_val(string_val); int rc = zstr_send(sock, string); CAMLreturn (Val_int(rc)); }  I can send and receive messages using these bindings in most of my code just fine. However, I have a scenario where I would like to do sends and receives inside of a signal handler, to the end of doing message passing in the background of some other code. Take this simplified example: open ZMQ exception SocketBindFailure let bg_ctx = zctx_new ();; let pub_sock = zsocket_new bg_ctx ZMQ_PUB;; let handler _ = print_endline "enter handler"; print_endline (string_of_int (zstr_send pub_sock "hello")); print_endline "end handler"; ;; let () = (try ( (* bind pub socket *) let rc = zsocket_bind pub_sock "tcp://*:5556" in if (rc < 0) then ( raise SocketBindFailure ); Sys.set_signal Sys.sigalrm (Sys.Signal_handle handler); ignore (Unix.setitimer Unix.ITIMER_REAL { Unix.it_interval = 0.01 ; Unix.it_value = 0.01 }); (* do some work *) ) with | SocketBindFailure -> raise SocketBindFailure) ;;  From the toplevel, this fails with the output: enter handler 0 end handler Fatal error: exception Sys_blocked_io  C code similar to the OCaml above works just fine. What is OCaml adding to the equation that is causing this exception? #### error message with scala installation I would like to install scala 2.8.1. I used exactly all the steps from http://www.scala-lang.org/node/310 but when i write 'scala' in the command prompt then i have the following error 'java' is not recognized as an internal or external command operable program or batch file Do you know whats going wrong? ### Portland Pattern Repository #### Silence Implies Consent (by d58-111-186-52.meb802.vic.optusnet.com.au 24 hours ago) ### StackOverflow #### LibGDX project written in Scala, on Android, using IntelliJ So I've been working on a LibGDX project in IntelliJ v12.1.6 using Scala after getting set up using this tutorial: http://www.gamefromscratch.com/post/2013/12/01/Using-LibGDX-with-IntelliJ-with-Gradle.aspx. However I've noticed, when testing the game on my android device, I receive the following error 12-04 17:46:22.483: ERROR/AndroidRuntime(18951): FATAL EXCEPTION: GLThread 475 Process: com.voliant.rsoss, PID: 18951 java.lang.NoClassDefFoundError: scala.Predef$
at com.voliant.rsoss.Actors.Player.act(Player.scala:20)
at com.voliant.rsoss.Screens.GameScreen.render(GameScreen.scala:46)
at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1523) at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1240)


Whenever I attempt to utilize a method that's native to Scala (such as the standard println() method).

I suspected that the android class could not find the classes utilized by Scala, so I attempted to export the android project with the Scala library. Whenever I try this however, I get an error that I have too many methods, (similar to this) which would make sense as Scala introduces well over enough methods into the project to warrant this error.

My question is how could I make the android project aware of the Scala classes and methods, so that I could use them as effectively in the android project as I have been able to in the desktop project?

Thanks!

#### Slick left/right/outer joins with Option

In the Slick examples there are a few examples of joining where one of the resulting columns can be nulls, as it can be the case when doing left, right, or outer joins. For example:

val explicitLeftOuterJoin = for {
(c, s) <- Coffees leftJoin Suppliers on (_.supID === _.id)
} yield (c.name, s.name.?)


But what if I want to return the entire mapped object? What I mean is:

val explicitLeftOuterJoin = for {
(c, s) <- Coffees leftJoin Suppliers on (_.supID === _.id)
} yield (c, s.?)


This doesn't seem to work as it complains about "could not find implicit value for evidence parameter of type scala.slick.lifted.TypeMapper[Suppliers]". Basically I'd like it to return a list of tuple of (Coffee, Option[Supplier])

Why doesn't this work and what's the fix for it? Especially, since this works fine:

val q = for {
c <- Coffees
s <- Suppliers
} yield (c, s)


(I know that's an inner join)

### QuantOverflow

#### Solvency II Assets-D1 - aggregation of line items by CIC code

The revised version of Assets D1 (now S.06.02 – List of assets under CP13) requires the following aggregations (page 24):

• CIC 71 (Cash), only one line per currency is to be reported;
• CIC 72 (Transferable deposits (cash equivalents)), only one line per pair(bank, currency) is to be reported;
• CIC 73 (Other deposits short term (less than one year)), only one line per pair(bank, currency) is to be reported;

CIC codes however are prefixed with a country code. So my question is - should this report have just ONE line per currency, or one line per currency per XX71 code? (and similarly for the other points..). If the former, then how should the CIC code be represented? (presumably 'CIC71').

EDIT: I don't see how this can be 'one line per currency' - as we also need to split between Life / Nonlife and RFFs.. Also, would 'look through' cash balances be aggregated too?

### CompsciOverflow

#### How to use Sigma Scaling in a GA

I have a genetic algorithm in java, I'm testing new types of selections.

For my tests I'm using the De Jong Half Sphere, my fitness function is :

x^2 + y^2


The selection method used is Sigma scaling :

fscaled = S + (fraw −fmean)/2σ


Example using S=1

 - a) Fitness = 5.927822124 |  After Scaling = 0.900756351532265  | 17.74410992 %
- b) Fitness = 3.431749363 |  After Scaling = 0.5720553077878213 | 10.27246375 %
- c) Fitness = 8.428103101 |  After Scaling = 1.2300115637030606 | 25.22835279 %
- d) Fitness = 2.548825744 |  After Scaling = 0.455785494144756  | 7.629554871 %
- e) Fitness = 13.07076638 |  After Scaling = 1.841391282832097  | 39.12551868 %


I know that sigma scaling should be used with a proportional selection.

• Is this the right way to implement this type of selection?

• If I wanted to minimize the function instead of maximize it how can I accomplish it, should I subtract the mean to each fitness and get the percentage this way or is there a better way ?

Cheers

### StackOverflow

#### Reactivemongo serializing a map into a BSONDocument

I defined BSONDocumentWriters to map domain objects (case classes) to BSON documents to be persisted in MongoDB using the ReactiveMongo driver. Defining the writers is quite straight-forward for case classes (although tedious and error prone: I wish there was a Salat-like solution for that). However, I can't seem to be able to do the same for a Map[String,Any] (where the keys can be of type numeric, date or string). I found a code example that defines a generic writer (and reader) for a map:

implicit def MapWriter[V](implicit vw: BSONDocumentWriter[V]): BSONDocumentWriter[Map[String, V]] =
new BSONDocumentWriter[Map[String, V]] {
def write(map: Map[String, V]): BSONDocument = {
val elements = map.toStream.map { tuple =>
tuple._1 -> vw.write(tuple._2)
}
BSONDocument(elements)
}
}


but this does not work if there's no implicit BSONDocumentWriter for the type V, i.e., the snippet:

BSONDocument(
"_id" -> "asd",
"map" -> MapWriter[Any].write(Map("x" -> 1, "y" -> "2"))
)


does not compile:

could not find implicit value for parameter vw: reactivemongo.bson.BSONDocumentWriter[Any]
"map" -> MapWriter[Any].write(Map("x" -> 1, "y" -> "2"))
^


I thought maybe the writer should write to a BSONValue instead of a BSONDocument so I modified the example as follows:

implicit def ValueMapWriter[V](implicit vw: BSONWriter[V, BSONValue]): BSONDocumentWriter[Map[String, V]] =
new BSONDocumentWriter[Map[String, V]] {
def write(map: Map[String, V]): BSONDocument = {
val elements = map.toStream.map {
tuple =>
tuple._1 -> vw.write(tuple._2)
}
BSONDocument(elements)
}
}


And to keep things simple, I tried to use Int as the value type but again, the snippet:

BSONDocument(
"_id" -> "asd",
"map" -> ValueMapWriter[Int].write(Map("x" -> 1, "y" -> 2))
)


does not compile:

could not find implicit value for parameter vw: reactivemongo.bson.BSONWriter[Int,reactivemongo.bson.BSONValue]
"map" -> ValueMapWriter[Int].write(Map("x" -> 1, "y" -> 2))
^


If the above had worked I could have used some base class as the value type and defined its implicit writer.

I am not sure why this is happening and how I can solve it. Maybe I am missing something obvious? Ideas?

When I hit loadtemplate_button, the page gets reloaded with the correct selection but the dropdownbox resets to the first choice. I want the dropdownbox to be displaying the last choice. Please note that when I select something from the dropdownbox and hit newsensor_button immediately, it works but that's not what I want ;). Any help would much be appreciated!!!

sensors.scala.html:

@(agentmodels : List[AgentModel])

@main("test") {

<select name="agentModelSelect" id="agentmodel_select"> @for(agentmodel <- agentmodels) {
<option value="@agentmodel.id">@agentmodel.name</option>}
</select>
<a class="btn btn-lg btn-primary" id="newsensor_button">Create new sensor</a>

<script>

var url = "/simulations/" + $("#agentmodel_select").val() + "/sensors/";$("#loadtemplate_button").attr('href', url);
}
$("#agentmodel_select").change(loadTemplateUrl); document.getElementById('newsensor_button').onclick = newSensorUrl; function newSensorUrl() { var url = "/simulations/" +$("#agentmodel_select").val() + "/sensors/new";
$("#newsensor_button").attr('href', url); } this.newSensorUrl();$("#agentmodel_select").change(newSensorUrl);
</script>


Controller file:

public static Result showListOfSensorsOfAgentModel(Long id) {
return ok(sensors.render(AgentModel.all()));
}// showListOfSensorsOfAgentModel


routes file:

GET         /simulations/:id/sensors/   controllers.AgentModelController.showListOfSensorsOfAgentModel(id: Long)


#### How to generate several Enumerators out of a single Enumerator (partition, split, ..)

Is it possible to create several Enumerators out of a single Enumerator ? What I'm looking for is the equivalent of List.partition that returns a (List[A], List[A]) , such as

List().partition(_.age >= 18)


So ideally, I'd like to apply a transformation on an Enumerator that allows me to partition the data so that I get a (Enumerator[T], Enumerator[T]) pair.

Is that easily doable using Play's Iteratee API ?

The closest thing I found is the Enumeratee.grouped method, which allows to group inputs, but as far as I understand, if all I want is 2 partitions (groups), I'll have to consume the entire Enumeratorto get a result. But I'd like the resulting Enumerators to be fed asynchronously from the input.

## What I want:

I have a clojure program on a remote site, let's call it mccarthy. What I want to do is connect to a nrepl-ritz from my laptop, preferably using nrepl-ritz-jack-in. The jack in works fine for a local program, but doesn't seems to connect to a remote program.

## Attempt 1

C-x C-f on /mccarthy:code/program/project.clj

(require 'nrepl-ritz)

M-x nrepl-ritz-jack-in

## Result

Emacs appears to hang. If I go to the *nrepl-server* buffer, I see this:

Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.flatland.drip.Main.invoke(Main.java:117) at org.flatland.drip.Main.start(Main.java:88) at org.flatland.drip.Main.main(Main.java:64) Caused by: java.lang.AssertionError: Assert failed: project at leiningen.ritz_nrepl$start_jpda_server.invoke(ritz_nrepl.clj:23) at leiningen.ritz_nrepl$ritz_nrepl.doInvoke(ritz_nrepl.clj:95)

(and tons of other lines, too...)

I am using drip on my laptop, but not on mccarthy, so clearly nrepl-ritz-jack-in is not detecting that it's a remote file. Regular old nrepl-jack-in will work as expected in this case, however.

## Attempt 2

I also tried starting an nrepl-ritz using lein on mccarthy:

mattox@mccarthy$lein ritz-nrepl nREPL server started on port 42874  from laptop, I forward a port so local 42874 connects to mccarthy 42874 ssh -L 42874:localhost:42874 -N mccarthy Then, from my local emacs: (require 'nrepl-ritz) M-x nrepl Host: 127.0.0.1 Port 42874 This gives me a connection: ; nREPL 0.1.7-preview user>  So to test it out, I run M-x nrepl-ritz-threads It gives me a nice table of threads. M-x nrepl-ritz-break-on-exception user> (/ 1 0)  ## Result This hangs, but sometimes shows a hidden debugging buffer with some restarts available. If I tell it to pass the exception back to the program, it never gives control back to the repl. I've done plenty of searches but have not been able to get anything more specific than "make sure lein is on your path". (And I did do that, on both machines...) Thanks for reading a (I hope not too) long question! ### CompsciOverflow #### Independent set to subset sum In http://pine.cs.yale.edu/pinewiki/SubsetSumReduction, could someone clarify what does$E$stand for in the target sum$\sum_{i\in E}b^i+k$? If$E$refer to$\underline{all\mbox{ }the\mbox{ }edges\mbox{ }in\mbox{ }the\mbox{ }graph}$, then the reduction from independent set to subset sum seems incorrect? ### TheoryOverflow #### Random 3-SAT: What is the consensus experimental range of the threshold? The critical ratio of clauses to variables for random 3-SAT is more than 3 and less than 6, and seems to be commonly described as "around 4.2" or "around 4.25". Mezard, Parisi, and Zecchina prove (in the physics sense) that the critical ratio is 4.256, whereas the first and third authors prove that it is 4.267. What is the range of values that the critical ratio could possibly take?  The motivation for me asking this question is that if the ratio could be$2+\sqrt{5} \approx 4.236$, then the standard reduction of 3-SAT to NAE-3-SAT (transforming$m$clauses and$n$variables into$2m$clauses and$m+n+1$variables) gives a ratio of$\phi$, which seems unlikely but would be pretty cool. ### StackOverflow #### Difference between clojure and clojurescript? Can anyone please list some differences between Clojure and ClojureScript? (Please and thank you) Syntax wise differences? ### /r/netsec #### Whois... Like A Boss! ### StackOverflow #### ZeroMQ: Publish to multiple Workers and wait for ACK I'm working on an app that notifies multiple Workers when a reboot is about to happen and then waits for ALL the Workers to do perform some tasks then send an ACK before rebooting. The number of Workers can change so somehow my app will need to know how many Workers are currently subscribed so that it knows that every Worker has sent an ACK. Is a pub/sub approach the best way for doing this? Does it provide a way to figure out how many subscribers are currently connected? Should my app use a REP socket to listen for ACK from the Workers? Is there a more elegant way of designing this? Thanks ### /r/netsec #### The Internet of Everything, Including Malware - Multi-architecture Linux worm spreading via PHP vulnerability ### StackOverflow #### How to display {} in hiccup? I am working on an angular app, where I need to display some angular expression in this form: {{"Hello," name}}  In Hiccup, {} have a special meaning, and is used for attributes, how to use it for angular syntax ? #### IntelliJ IDEA 13 Scala Plugin & Eclipse Kepler Compilation Error Code : import com.google.common.base.Joiner import scala.collection.JavaConverters._ object MainJ extends App { val fantasyGeneres = Array("Space Opera", "Horror", "Magic realism", "Religion") val joined = Joiner.on(',').join(fantasyGeneres.toIterable.asJava) println(joined) }  IntelliJ IDEA Error : scala: error while loading Joiner, class file 'C:\guava- 15.0.jar(com/google/common/base/Joiner.class)' is broken (class java.lang.RuntimeException/bad constant pool tag 9 at byte 10)  Scala Plugin Version : Scala -> 0.26.318 Version: 0.26.318 Date: 2013-12-04 10:46 Since: 133.124 Until: 134.0 Size: 25346.32 Kb  Eclipse Error : Description Resource Path Location Type error while loading Joiner, class file 'C:\guava-15.0.jar(com/google/common/base/Joiner.class)' is broken (class java.lang.RuntimeException/bad constant pool tag 9 at byte 10) Scala Unknown Scala Problem  please see above images, I wrote a program with Guava library and scala language WHEN I use Eclipse IDE with Scala Plugin & IntelliJ IDEA with Scala Plugin above error shows and I cant compile it Can anyone help me ? ### CompsciOverflow #### Ball Break Point Problem I came across following question: There is a building with N floors. We have a balls of same property and have a breaking point of X floor. i.e, If ball dropped from X+1 floor then the ball will break. if ball is dropped from X or less than X then it wont break. Now design an algorithm to find X if N is given. My Solution: Drop the ball from N/2 floor. If ball breaks then X < N/2. Else X > N/2. Repeat this process till X is found. but If X is N/2+1 or N/2-1 the number of steps to find X are high. Is there a better way to Find X? ### QuantOverflow #### How does Interactive Broker's historical data compare to other alternatives? Has any of you stumbled with shows stopper problems like it having survivorship bias? ### StackOverflow #### Accessing only part of a list in Scala using .last I have the following code which val result = hyp().filter(_._3)  where hyp() returns List[(Int,Int,Set[Int])] and filter is used to return only those Lists where the set contains single element. _._3 accesses the Set(Int) part. How can I use .last instead of _._3? Thanks #### clojure contrib logging to file Hi I have tried logging with clojure contrib but to no avail. Its clojure 1.1 (ns com.class.main.service (:gen-class) (:require (clojure.contrib [logging :as log]))) (log/info "Hello world")  I have put a log4j.jar file in my bin and a log4j.properties in the same directory as the log4j.jar. Below is the log4j.properties conifguration log4j.rootLogger=INFO, file log4j.appender.file=org.apache.log4j.RollingFileAppender log4j.appender.file.File=H:\\log\\loging.log log4j.appender.file.MaxFileSize=1MB log4j.appender.file.MaxBackupIndex=1 log4j.appender.file.layout=org.apache.log4j.PatternLayout log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n*  I have gone through some basic examples although they are limited due to most using the newer version with tool.logging. But below are the examples i have used https://github.com/macourtney/Conjure/wiki/How-to-use-logging & http://www.paullegato.com/blog/setting-clojure-log-level/ have helped me get this far the log prints out on the console of emacs but i can't find a log file anywhere. ### UnixOverflow #### Does any OS support ZFS on MIPS? Does there exist an operating system supporting ZFS under the MIPS architecture? If so, which operating system(s)? I am especially interested in: ### TheoryOverflow #### Depth First Search mark node as unvisited? [on hold] I am reading Top Coder tutorial on Graphs, and the algorithm for DFS is given as follows: dfs(node start) { stack s; s.push(start); while (s.empty() == false) { top = s.top(); s.pop(); mark top as visited; check for termination condition add all of top's unvisited neighbors to the stack. mark top as not visited; } }  I have never seen mark top as not visited; step as a part of dfs. Why is it necessary? Isn't the point of dfs to visit all the nodes on the graph? why then mark already visited one as non-visited? ### StackOverflow #### Decimal expansions using lazy sequences in Clojure Is there a package which represents decimal expansions in Clojure using lazy sequences? For example, syntax like (defn r (B N x_1 x_2 x_3 ...))  could represent a real number r in base B, with decimal expansion (in math notation) r = N . x_1 x_2 x_3 ...  with integer significand N and decimal digits 0 ≤ x_i ≤ B-1. If the type were "smart" enough, it could handle different decimal expansions of real numbers as valid inputs, such as (10 0 9 9 9 ...) and (10 1), and consistently output decimal expansions in the latter form. It should also be able to handle overflowing digits, like reducing (10 0 15) to (10 1 5). Is there any obstruction to working with a lazy-sequence representation of real numbers instead of the usual decimal expansion? I don't know how efficient it would be in contrast to floating-point, but it would be convenient for doing rigorous precise arithmetic involving real numbers. For example, I think there are algorithms which recursively compute the decimal expansions of π and e. ### CompsciOverflow #### Equivalence of two lambda expressions for NOT I've seen two different lambda expressions for the logical NOT function. One of them just applies its parameter to constants true and false internally in a reverse order:$NOT \;\;=\;\; \lambda x.\; x \; \mbox{false}\; \mbox{true} \;\;=\;\; \lambda x. \; x \; (\lambda t.\lambda f.f) \; (\lambda t. \lambda f. t ) $and the other one which captures two more parameters instead of passing them to the returned function externally, and applies$x$to them in reverse order too:$NOT \;\;=\;\; \lambda x. \; \lambda t.\lambda f.xft$of which the other one seems a bit simpler (and it is also simpler when encoded in binary). So my question is this: Is there any transformation which could get me from one of these representations to the other? I see they are equivalent "extensionally", that is, both produce the same results. But I'd like to "prove" it somehow by algebraic transformations, such as those for alpha, beta and eta conversions. Unfortunately, none of these can help me in this case: Alpha is just for renaming. Beta works only for function calls, but we don't have any function call here which could be reduced, since$x$is free in the function body (not the whole expression, though) in all these expressions until we actually call NOT on something. The most close one seems to be the eta, which is related to extensional equivalence and forwarding parameters, but when the parameters are being reversed, it's not a simple forwarding anymore and eta doesn't seem to apply here. Is there any more conversion rule I'm missing? (Well, I guess they won't just skip two Greek letters from no particular reason, wouldn't they?) P.S.: This question is actually a model one, since there are many other definitions for other functions which have several different forms which seem to be extensionally equivalent, but completely different with regard to the known reduction rules. I've chosen the simplest example of this problem. Edit: To better clarify what I'd like to know, here's a diagram showing the reduction steps for both versions of the not function: http://sasq.comyr.com/Stuff/Lambda/Not.png As you can see, they really both reduce to the same results (contrary to what @Jonathan Gallagher was saying below). This is what I already know: that they are confluent, and so they're Church-Rosser equivalent. What I don't know however is if there is any conversion rule (similar to alpha, beta & eta) which could allow me to transform one form of not to the other. This would allow me to at least make sure if some other functions (more complicated than these two here), are also equivalent, which could be hard when they can reduce to more than just two possible answers. I'd like to know if there's some conversion rule which could allow me to convert one intensional definition into another when two functions are already extensionally equivalent (i.e. they give the same results for the same parameters). Or (which would be better) if some function can be converted somehow to the other even if I don't know whether they're extensionally equivalent. ### /r/compsci #### [Question for professors] What is an appropriate/acceptable gift from a student? One of my departmental advisers has really gone out of the way to help me this year and make sure I graduate on time and I just wanted to show my appreciation by getting him a small gift now that we're nearing the holidays. My initial thought was a six-pack of good beer (everyone loves beer, right?), but I don't know if I should really just roll into his office with a wrapped up case of alcohol. Do you guys have any ideas for something that would be cool and appropriate for being dropped off during a workday? submitted by lucem [link] [1 comment] ### /r/netsec #### Good Practice Guide for securely deploying Governmental Clouds ### StackOverflow #### Read first X bytes of input stream in Scala What is the most compact and scala-like way to read the first X bytes of an input stream to a buffer? #### How to show tree navigation menu in scala? I'm a newbie in scala. I am trying to build navigation tree like • Parent • Child1 • Child2 • Child3 • Child4 I have a model class for it. case class Field(id:Long, name:String, icon:String, parentid:Long) object Field { def all():List[Field] = { List( Field(1,"Parent1","1", 0), Field(2,"Child1","2",1), Field(3,"Child2","3",1), Field(4,"Child3","4",1), Field(5,"Child4","5",1) ) } }  And in my view I have a code for rendering: @fields.groupBy(_.parentid).map { case ( parentid, tasks) => <ul> <li>@parentid</li> <ul> @tasks.map { task => <li>@task.name</li> } </ul> </ul> }  But unfortunately the output is: • 1 • Child1 • Child2 • Child3 • Child4 • 0 • Parent1 How to build navigation menu, where model has child/parent relationship? I hope you understand my question, and will help me. Sorry for my bad english #### The strange case of multiple Futures in Scala What's the the connection between those Future-related class and traits in Scala, and why are they sprinkled over different packages? I have found those: abstract class scala.actors.Future object scala.actors.Futures trait/object scala.collection.parallel.FutureThreadPoolTasks trait scala.concurrent.FutureTaskRunner trait scala.parallel.Future (the package consists of only this file...)  Do they significantly different things or is there another reason why they can't be consolidated? Is there a good example showing when one would use the one thing or the other? Edit: Bounty for explaining what each of the classes/traits/objects does and how they justify their existance/how they are useful. #### Scala Run Command and Capture Output I have a string with the absolute path to a command: val path = "C:\\Program Files\\Some Folder\\runme.exe"  then I do the following to capture its output: path.!!  But I get an exception: java.io.IOException: Cannot run program "C:\Program" Is it because there is a space in the path? How Can I do this? ### CompsciOverflow #### How to prove that a language is not context-free? We learned about the class of context-free languages$\mathrm{CFL}$. It is characterised by both context-free grammars and pushdown automata so it is easy to show that a given language is context-free. How do I show the opposite, though? My TA has been adamant that in order to do so, we would have to show for all grammars (or automata) that they can not describe the language at hand. This seems like a big task! I have read about some pumping lemma but it looks really complicated. ### /r/emacs #### Problem with command-history with shell+tramp. ### Portland Pattern Repository #### Expressive Internet Architecture (by 99-98-229-88.lightspeed.mssnks.sbcglobal.net 27 hours ago) #### Choice Net (by 99-98-229-88.lightspeed.mssnks.sbcglobal.net 27 hours ago) ### CompsciOverflow #### Construct a deterministic finite automation The question asks to: construct a DFA which accepts exactly$\frac{n(n-1)(n-2)}{6} + \frac{n(n-1)}{2}+1$many members of$\{0, 1\}^n$for every n. I have no idea where to start to constructing the DFA, could you give some directions? By the way, how many states should this DFA have? ### /r/compsci #### Homotopy type theory lectures and notes on-line ### /r/scala #### Trait DI vs Constructor DI Hi I'm interested to know what exactly is the "idiomatic" approach to DI in Scala. I have been doing Scala a while and use "old fashioned" DI, which is just via the constructor: trait MyTrait{ def say: String } class ConcreteImpl extends MyTrait{ def say = "Hello, world" } class TestImpl extends MyTrait{ def say = "Testing" } class Foo(thing: MyTrait){ println(thing.say) } val x = new Foo(new ConcreteImpl) val y = new Foo(new TestImp)  But I often see examples like the following, using traits trait MyTrait{ def say: String } trait ConcreteTrait extends MyTrait{ def say = "Hello, world" } trait TestTrait extends MyTrait{ def say = "Testing" } class Foo{ this: MyTrait => println(say) } val x = new Foo // compilation error val y = new Foo with ConcreteTrait // works val test = new Foo with TestTrait // for testing  Both of these approaches achieve the same thing; is there a preferred idiomatic approach? And if so, why is one better than the other? submitted by quiI [link] [13 comments] ### StackOverflow #### Scope management - Stateful IO Monad? I am playing with functional programming and in particular with Functional Java. I have implemented with success my version of the IO Monad and I am writing IO actions for my core. It is basically serializing objects to Xml files (the object type extends the custom XmlWritable interface). Unfortunately, in order to do that, an instance of OutputStream AND one instance of XmlSerializer needs to be created. The scope of the OutputStream is wider than XmlSerializer's, which means that the only way I can see to be able to correctly handle both lifecycles within my IO monad is to carry both of them with me in a tuple, closing OutputStream after having written using XmlSerializer. This leads to heavy and ugly code (Java 6 is definitely not the best for this): public abstract class IO<R> { [...] } public class IOActions { public final F<String, IO<OutputStream>> openFileFn() { return new F<String, IO<OutputStream>>() { @Override public IO<OutputStream> f(String fileName) { [...] } }; } /* This will be partially applied, encoding will be fixed */ public static final F<OutputStream, IO<P2<OutputStream, XmlSerializer>>> initSerializer() { return new F<OutputStream, IO<P2<OutputStream, XmlSerializer>>>() { @Override public IO<P2<OutputStream, XmlSerializer>> f(OutputStream os) { XmlSerializer = new ... [...] } }; } /* This will be partially applied as well */ public static final F2<XmlWritable, P2<OutputStream, XmlSerializer>, IO<P2<OutputStream, XmlSerializer>>> writeObjectFn() { return new F2<XmlWritable, P2<OutputStream, XmlSerializer>, IO<P2<OutputStream, XmlSerializer>>>() { @Override public IO<P2<OutputStream, XmlSerializer>> f(XmlWritable object, P2<OutputStream, XmlSerializer> p) { [...] } }; }  Is there a more idiomatic why to handle my use case in functional programming? Lurking, I discovered the State Monad...but I am kind of scared to see what it is going to happen if I apply a State Monad on top of a IO Monad in Functional Java. ### CompsciOverflow #### nlogn algorithm for geometric problem What is an$O(n \times log(n))algorithm to find how big is the largest subset of n axis-aligned rectangles (in the plane) that contain a common point? Perhaps by reducing this to a problem with such runtime? AnO(n^{3})$algorithm could be for each of the n rectangles for each of the 4 sides of the rectangle find the points where it intersects other rectangles and keep the set of those points loop through the intersection points set and update the count  ### StackOverflow #### Mailbox abstraction concurrent ml I am trying to implement a mailbox abstraction in Concurrent ML. The mailbox maintains a list containing values from inChan and values are sent to outChan which other threads can consume. Here is the code I have so far: fun mailbox inChan outChan list = let val inChan=channel() val outChan=channel() fun loop xs= fn i=> if i<=0 (recvEvt inChan ; loop (tl xs) ) else (sendEvt(outChan,x); loop (i::xs) ) in spawn(fn()=>loop xs); end  Would this be okay? ### Portland Pattern Repository #### Nebula Cloud (by 99-98-229-88.lightspeed.mssnks.sbcglobal.net 28 hours ago) #### Future Internet Architecture Program (by 99-98-229-88.lightspeed.mssnks.sbcglobal.net 28 hours ago) #### Mobility First (by 99-98-229-88.lightspeed.mssnks.sbcglobal.net 28 hours ago) #### National Science Foundation (by 99-98-229-88.lightspeed.mssnks.sbcglobal.net 28 hours ago) ### TheoryOverflow #### Computational Model in SETH Impagliazzo, Paturi and Calabro, Impagliazzo, Paturi introduced Exponential-Time Hypothesis (ETH) and Strongly Exponential-Time Hypothesis (SETH). Roughly, SETH says that there is no algorithm which solves SAT in time$1.99^n$. I was wondering what would that mean to break SETH. We definitely need to find an algorithm which solves SAT in fewer than$2^n$steps, but I don't quite understand what computational model we should use. As far as I know, results based on SETH (see, e.g., Cygan, Dell, Lokshtanov, Marx, Nederlof, Okamoto, Paturi, Saurabh, Wahlstrom) don't need to make assumptions about the underlying model of computation. Assume, for example, that we found an algorithm which solves SAT in time$1.5^n$using space$1.5^n$. Does it automatically imply that we can find a Turing Machine which solves this problem in time$1.99^n$? Does it break SETH? ### StackOverflow #### How to make this functional-style code more performant while maintaining its syntax I would like to write this operation Matrix.Multiply(source,target) target.Add(offsets)  in this mathematical style target = Matrix * source + offsets;  However, I'm also concerned about performance. (In fact, the above operation will be run in a tight loop.) I don't want to create a new vector for each matrix-vector multiplication and another new vector for the vector addition. How can I implement the above operation in C# or F# (or any functional language that you know) so that style and performance can be both achieved? Sorry for this naive question. I'm sure this desire is so fundamental. ### Lobsters #### F# Data: JSON Type Provider ### CompsciOverflow #### Turing decidable/undecidable let$X = \{\langle M \rangle\ |\ M\text{ is a finite state machine and }L(M) = \emptyset\}$where$\langle M \rangle$is an encoding of the machine$M$. can you prove whether$X$is Turing decidable/undecidable? this question could be on my exam this week and I need to understand it. please help. #### "Unusual" coupling between a decision problem and a corresponding optimization problem There seems to usually be a tight connection between decision problems and (corresponding) optimization problems in general. However, is this always the case? Are there examples where the typical "tight coupling" between a decision problem and the correponding optimization problem breaks down or behaves in an unusual way, e.g. have significantly different complexity? Or, maybe there is a case where there is a cluster of problems that are all closely related, but the "best" or "definitive" version is not obvious or apparent? Also, I am looking for any survey or broad overview or discussion of this apparent basic connection between decision and optimization problems. A similar question was asked here, but the answers were highly theoretical and it did not seem to yield any specific or tangible examples. #### Visualize Graph Clusters I am working on my thesis which involves using ant based techniques for graph clustering. I am testing the algorithm currently and I was wondering if there is a way that I can visualize the clusters of a given graph. In this case, I have the graph as a file and another file corresponding to the clustering where each node is given its cluster. Is there a software using which I can visualize this graph where nodes belonging to the same cluster have like the same color or something? The graph is represented as an edge list, or in the ".net" format or Graph Modelling Language ".gml" format. I know there are a lot of programs out there for networks such as gephi, graphviz, networkx, pajek etc but I don't know how to do this. Any help/ideas would be greatly appreciated! Thanks! ### Lobsters #### How I Landed a Job Offer After 18 Weeks Learning to Code ### /r/dependent_types #### Homotopy Type Theory course (with lecture notes and videos) ### Portland Pattern Repository #### This One Has Three (by 99-98-229-88.lightspeed.mssnks.sbcglobal.net 29 hours ago) ### StackOverflow #### Scala match on generic type is it possible to match somehow on the return type A and if it is e.g. an int, do a calculation that returns an int. See the following example: def test[A](a: A):A = a match{ case b: Int => b * 5 case _ => a }  The error message: type mismatch; found : Int required: A  Thank you! ### Lobsters #### sysadvent: Day 4 - Yum and repository tools by example ### CompsciOverflow #### prove the halting problem is undecidable Prove that it is undecidable for the halting problem of a deterministic Turing machine which accepts nonrecursive language or in-other-words: let's say we have a deterministic Turing machine which accepts nonrecursive language, prove the halting problem is undecidable I tried to prove it saying that since it accept nonrecursive language then it must be accepting R.E. languages where it's not guaranteed to be decidable. therefore it's undecidable. apparently this does not make sense to everyone. please help if you can prove it and thanks in advance :) ### Lobsters #### Measuring C API Coverage with Go ### StackOverflow #### SBT Native Packager - How to retrieve libraryDependencies and store them as part of the package? I recently discovered the SBT Native Packager plugin which I thought would help solve all my packaging and deployment problems. On the face of it, it looks like it should however I have a problem which I would assume is a fairly basic ask; I want to be able to copy all managed dependencies from my project to the lib directory of my package for deployment. I read through the docs carefully but there doesn't seem to be an obvious way to do this. There are some instructions in there that explain how to copy the artefact from the project: mappings in Universal <+= (packageBin in Compile) map { jar => jar -> ("lib/" + jar.getName) }  but I want to be able to do the same for the managed dependencies. Did I miss something? I read through most of the sbt docs on managed dependencies and that didn't give me the answer either. I could of course use retrieveManaged := true to download into the lib_managed dir and grab from there but it feels like a bit of a hack. Can anyone help? James ### Lobsters #### How does the NSA break SSL? ### CompsciOverflow #### Example of an algorithm where a low-order term dominates the runtime for any practical input? Big-O notation hides constant factors, so some$O(n)$algorithms exist that are infeasible for any reasonable input size because the coefficient on the$n$term is so huge. Are there any known algorithms whose runtime is$O(f(n))$but with some low-order$o(f(n))$term that is so huge that for reasonable input sizes it completely dominates the runtime? I'd like to use an algorithm like this an an example in an algorithms course, as it gives a good reason why big-O notation isn't everthing. Thanks! ### Lobsters #### No, I will not hack on your codebase for free under the pretext of it being a "technical interview" A little hyperbolic, but a well-made point otherwise. Comments #### Subverting (UEFI) security with kexec ### /r/emacs #### EmacsWiki: FrequentlyEnabledPackages Emacs244 Survey ### StackOverflow #### Concurrent ML Mailbox abstraction I am trying to implement a mailbox abstraction in Concurrent ML. The mailbox maintains a list containing values from inChan and values are sent to outChan which other threads can consume. Here is the code I have so far: fun mailbox inChan outChan list = let val inChan=channel() val outChan=channel() fun loop x= sync(choose[ (recvEvt inChan ; list@[recv inChan] ; loop ), (sendEvt(outChan,x);loop x ) ] in spawn(fn()=>loop x); inChan outChan end  #### Scala implicits resolution mechanism is declaration order dependent? During daily Scala coding I faced an issue that Scala implicits resolution depends on declaration order. A simple example: object example extends App { trait FooTypeClass[T] { def foo: T } def bar[T](implicit tc: FooTypeClass[T]) = println(tc.foo) class A { // bar[A] doesn't compile } object A { implicit object aFoo extends FooTypeClass[A] { def foo: A = new A { override def toString = "a" } } } bar[A] }  It does compile, but if I uncomment the commented line, it won't find the required implicit in scope. So, to make it compile I should place object A declaration before class A. That means my companions should go before their classes but I'd rather leave them as they are in the example. Why does this order matter? Is there a workaround for this? ### Fefe #### Was wird eigentlich aus dem Geheimdienst-Mascolo? ... Was wird eigentlich aus dem Geheimdienst-Mascolo? Der geht jetzt zu Axel Springer. ### /r/netsec #### PayPal Phishing Text ### StackOverflow #### Can't setup typesafe console I'm tring to enable typesafe console for my akka+spray based server, but it doesn't work. I've successfully added sbt-atmos plugin to my sbt build (as it's recommended here), when I run atmos:run it starts without warnings, I can see console web interface on localhost:9000, but, unfortunately, when I send some requests to my server, nothing changes. Console shows 0 nodes. My application is an http server, it uses akka 2.2.3, sbt 0.13.0 and spray 1.2.0. In my Build.scala I have: lazy val SomeProject = Project( id = "SomeProject", base = file("."), fork := true, libraryDependencies ++= Dependencies.someProject, resolvers ++= Dependencies.someProjectResolvers, distJvmOptions in Dist := "-Xms256M -Xmx1024M", outputDirectory in Dist := file("some-proj-dist"), javaOptions := Seq("-XX:+CMSClassUnloadingEnabled", "-XX:+UseConcMarkSweepGC", "-XX:MaxPermSize=512M", "-XX:PermSize=64M", "-Xmx2048M", "-XX:ReservedCodeCacheSize=256M")//, ) ++ PB.protobufSettings ++ Revolver.settings ).configs(Atmos).settings(atmosSettings: _*)  Dependency object look like this: object Dependency { // Versions object V { val Akka = "2.2.3" val ScalaTest = "2.0" } val akkaKernel = "com.typesafe.akka" %% "akka-kernel" % V.Akka val akkaSlf4j = "com.typesafe.akka" %% "akka-slf4j" % V.Akka val logback = "ch.qos.logback" % "logback-classic" % "1.0.0" val akkaActor = "com.typesafe.akka" %% "akka-actor" % V.Akka val akkaTest = "com.typesafe.akka" %% "akka-testkit" % V.Akka % "test" val scalaTest = "org.scalatest" %% "scalatest" % V.ScalaTest % "test" val sprayRouting = "io.spray" % "spray-routing" % "1.2.0" val sprayCan = "io.spray" % "spray-can" % "1.2.0" val sprayJson = "io.spray" %% "spray-json" % "1.2.5" val protobuf = "com.google.protobuf" % "protobuf-java" % "2.5.0" val scalaCheck = "org.scalacheck" %% "scalacheck" % "1.11.0" % "test" }  Can please anybody help my to find out what's wrong? By the way, can typesafe console work with akka 2.2.3? typesafe activator has 2.2.1 as far as I know.. Also, I see a strange warning in atmos log: WARN [U] [ActorSystem(atmos)] [pool-1-thread-1] : Couldn't accept new trace receiver connection - already at max connections  I tried to google it, but wasn't able to find anything about it. What does it mean? ### High Scalability #### How Can Batching Requests Actually Reduce Latency? Jeremy Edberg gave a talk on Scaling Reddit from 1 Million to 1 Billion–Pitfalls and Lessons and one of the issues they had was that they: Did not account for increased latency after moving to EC2. In the datacenter they had submillisecond access between machines so it was possible to make a 1000 calls to memache for one page load. Not so on EC2. Memcache access times increased 10x to a millisecond which made their old approach unusable. Fix was to batch calls to memcache so a large number of gets are in one request. Dave Pacheco had an interesting question about batching requests and its impact on latency: I was confused about the memcached problem after moving to the cloud. I understand why network latency may have gone from submillisecond to milliseconds, but how could you improve latency by batching requests? Shouldn't that improve efficiency, not latency, at the possible expense of latency (since some requests will wait on the client as they get batched)? Jeremy cleared it up by saying: The latency didn't get better, but what happened is that instead of having to make a lot of calls to memcache it was just one (well, just a few), so while that one took longer, the total time was much less. But Dave Rosenthal created a great graphic showing how batching can in fact decrease total system latency: ### StackOverflow #### How to Debug ClojureScript I apologize for this seemingly stupid question, but I've been playing with ClojureScript on and off for a few weeks now, and I can't figure out this one simple question: How do I debug ClojureScript? So here is the problem: 1. I write my *.cjs files 2. I run cljsc/build ... 3. I load up my webpage. 4. Something bad happens. 5. I open up the firefox console. 6. I get a line in the generated js, which I find incomprehensible, and I have no idea which line of the original cljs file it came from. My question: What is the right way to develop ClojureScript applications? PS I've looked at ClojureScriptOne -- what I don't like about it is that it strings together a bunch of technology all at once; and I'd prefer to understand how to use each individual piece on its own before chaining it all together. I'm comfortable with ring + moustache + compojure, [mainly because I can use my standard Clojure debugging techniques] but ClojureScript is another beast. UPDATE: Things have changed quite a bit since this question was first asked. The proper way to debug ClojureScript applications these days is to enable source maps - http://github.com/clojure/clojurescript/wiki/Source-maps #### Can ZeroMQ be used to accept traditional socket requests? I'm trying to re-write one of our old Servers using ZeroMQ, for now I have the following Server setup, (which works for Zmq requests):  using (var context = ZmqContext.Create()) using (var server = context.CreateSocket(SocketType.REP)) { server.Bind("tcp://x.x.x.x:5705"); while (true) { ... }  This kind of setup works fine if I use the Zmq client library to connect context.CreateSocket(SocketType.REQ) But unfortunately we've got a lot of legacy code that needs to connect to this server and the sockets are created using .net socket libs:  Socket = new Socket(ipAddress.AddressFamily, SocketType.Stream, ProtocolType.Tcp); Socket.Connect(ipAddress, port);  Is there a way to write a ZeroMQ Server to accept these traditional .net socket connections? ### /r/netsec #### A Security Analysis of Computer Tablets Aimed at Children ### Planet Theory #### Power Method still Powerful The Power Method is the oldest practical method for finding the eigenvector of a matrix corresponding to the eigenvalue of largest modulus. Most of us will live to celebrate the Power Method’s 100th birthday around 2029. I plan to throw a party. What’s surprising is that the Power Method continues to be highly relevant in a number of recent applications in algorithms and machine learning. I see primarily two reasons for this. One is computational efficiency. The Power Method happens to be highly efficient both in terms of memory footprint and running time. A few iterations often suffice to find a good approximate solution. The second less appreciated but no less important aspect is robustness. The Power Method is quite well-behaved in the presence of noise making it an ideal candidate for a number of applications. It’s also easy to modify and to extend it without breaking it. Robust algorithms live longer. The Power Method is a great example. We saw another good one before. (Hint: Gradient Descent) In this post I’ll describe a very useful robust analysis of the Power Method that is not as widely known as it sould be. It characterizes the convergence behavior of the algorithm in a strong noise model and comes with a neat geometric intuition. ## Power Method: Basics Let’s start from scratch. The problem that the power method solves is this: Given ${A,}$ find the rank ${1}$ matrix ${B}$ that minimizes ${\|A-B\|_F^2.}$ Here, ${\|A-B\|_F^2}$ is the squared Frobenius norm of the difference between ${A}$ and ${B.}$ (Recall that this is the sum of squared entries of $A-B$). Any rank ${1}$ matrix can of course be written as ${xy^T}$ (allowing for rectangular ${A}$). On the face of it, this is a non-convex optimization problem and there is no a priori reason that it should be easy! Still, if we think of ${f(x,y) = \|A-xy^T\|_F^2}$ as our objective function, we can check that setting the gradient of ${f}$ with respect to ${x}$ to zero gives us the equation by ${x=Ay}$ provided that we normalized ${y}$ to be a unit vector. Likewise taking the gradient of ${f}$ with respect to ${y}$ after normalizing ${x}$ to have unit norm leads to the equation $y={A^T x.}$ The Power Method is the simple algorithm which repeatedly performs these update steps. As an aside, this idea of solving a non-convex low-rank optimization problem via alternating minimization steps is a powerful heuristic for a number of related problems. For simplicity from here on we’ll assume that ${A}$ is symmetric. In this case there is only one update rule: ${x_{t+1} = Ax_{t}.}$ It is also prudent to normalize ${x_t}$ after each step by dividing the vector by its norm. The same derivation of the Power Method applies to the more general problem: Given a symmetric ${n\times n}$ matrix ${A,}$ find the rank ${k}$ matrix ${B}$ that minimizes ${\|A-B\|_F^2}$. Note that the optimal rank ${k}$ matrix is also given by the a truncated singular value decomposition of ${A.}$ Hence, the problem is equivalent to finding the first ${k}$ singular vectors of ${A.}$ A symmetric rank ${k}$ matrix can be written as ${B= XX^T}$ where ${X}$ is an ${n\times k}$ matrix. The update rule is therefore ${X_{t+1} = AX_{t}}$. There are multiple ways to normalize ${X_t}$. The one we choose here is to ensure that ${X_t}$ has orthonormal columns. That is each column has unit norm and is orthogonal to the other columns. This is can be done using good ol’ boys Gram-Schmidt (though in practice people prefer the Householder method). The resulting algorithm is often called Subspace Iteration. I like this name. It stresses the point that the only thing that will matter to us is the subspace spanned by the columns of ${X_t}$ and not the particular basis that the algorithm happens to produce. This viewpoint is important in the analysis of the algorithm that we’ll see next. ## A Robust Geometric Analysis of Subspace Iteration Most of us know one analysis of the power method. You write your vector ${x_t}$ in the eigenbasis of ${A}$ and keep track of how matrix vector multiplication manipulates the coefficients of the vector in this basis. It works and it’s simple, but it has a couple of issues. First, it doesn’t generalize easily to the case of Subspace Iteration. Second, it becomes messy in the presence of errors. So what is a better way? What I’m going to describe was possibly done by numerical analysts in the sixties before the community moved on to analyze more modern eigenvalue methods. ### Principal Angles To understand what happens in Subspace Iteration at each step, it’ll be helpful to have a potential function and to understand under which circumstances it decreases. As argued before the potential function should be basis free to avoid syntactic arguments. The tool we’re going to use is the principal angle between subspaces. Before we see a formal definition, it turns out that much of what’s true for the angle between two vectors can be lifted to the case of subspaces of equal dimension. In fact, much of what’s true for the angle between spaces of equal dimension generalizes to unequal dimensions with some extra care. The two subspaces we want to compare are the space spanned by the ${k}$ columns of ${X_t}$ and the space ${U}$ spanned by the ${r}$ dominant singular vectors of ${A.}$ For now, we’ll discuss the case where ${r=k.}$ I’ll be a bit sloppy in my notation and use the letters ${U,X_t}$ for these subspaces. Let’s start with ${k=1}$. Here the cosine of the angle ${\theta}$ between the two unit vectors ${X_t}$ and ${U}$ is of course defined as ${\cos\theta(U,X_t)= |U^T X_t|.}$ It turns out that the natural generalization for ${k>1}$ is to define $\displaystyle \cos\theta(U,X_t)=\sigma_{\mathrm{min}}(U^T X_t).$ In words, the cosine of the angle between ${U}$ and ${X_t}$ is the smallest singular value of the ${k\times k}$ matrix ${U^T X_t}$. Here’s a sanity check on our definition. If the smallest singular value is ${0}$ then there is a nonzero vector in the range of the matrix ${X_t}$ that is orthogonal to the range of ${U}$. In an intuitive sense this shows that the subspaces are quite dissimilar. To get some more intuition, consider a basis ${V}$ for the orthogonal complement of ${U.}$ It makes sense to define ${\sin\theta(U,X_t)= \sigma_{\mathrm{max}}(V^T X_t)}$. Indeed, we can convince ourselves that this satisfies the familiar rule ${1= \sin^2\theta + \cos^2\theta}$. Finally, we define ${\tan\theta}$ as the ratio of sine to cosine. ### A strong noise model Let’s be ambitious and analyze Subspace Iteration in a strong noise model. It turns out that this will not actually make the analysis a lot harder, but a lot more general. The noise model I like is one in which each matrix-vector product is followed by the addition of a possibly adversarial noise term. Specifically, the only way we can access the matrix $A$ is through an operation of the form $\displaystyle Y = AX + G.$ Here, ${X}$ is what we get to choose and ${G}$ is the noise matrix we may not get to choose. We assume that $G$ is the only source of error and that arithmetic is exact. In this model our algorithm proceeds as follows: Pick ${X_0.}$ For ${t = 1}$ to ${t=L:}$ 1. ${Y_t = AX_{t-1} + G_t}$ 2. ${X_t = \text{Orthonormalize}(Y_t)}$ ### The main geometric lemma To analyze the above algorithm, we will consider the potential function ${\tan\theta(U,X_t).}$ If we pick ${X_0}$ randomly it’s not so hard to show that it starts out being something like ${O(\sqrt{n/k})}$ with high probability. We’ll argue that ${\tan\theta(U,X_t)}$ decreases geometrically at the rate of ${\sigma_{k+1}/\sigma_{k}}$ under some assumptions on ${G_t}$. Here, ${\sigma_k}$ is the ${k}$-th largest singular value. The analysis therefore requires a separation between ${\sigma_k}$ and ${\sigma_{k+1}}$. (NB: We can get around that by taking the dimension ${r}$ of ${U}$ to be larger than ${k.}$ In that case we’d get a convergence rate of the form ${\sigma_{r+1}/\sigma_k.}$ The argument is a bit more complicated, but can be found here.) Now the lemma we’ll prove is this: Lemma. For every ${t\ge 1,}$ $\displaystyle \tan\theta(U,X_t) \le \frac{\sigma_{k+1}\sin\theta(U,X_{t-1})+\|V^T G_t\|} {\sigma_k\cos\theta(U,X_{t-1})-\|U^T G_t\|}.$ This lemma is quite neat. First of all if ${G_t=0,}$ then we recover the noise-free convergence rate of Subspace Iteration that I claimed above. Second what this lemma tells us is that the sine of the angle between ${U}$ and ${X_{t-1}}$ gets perturbed by the norm of the projection of ${G_t}$ onto ${V}$, the orthogonal complement of ${U.}$ This should not surprise us, since ${\sin\theta(U,X_{t-1})=\|V^T X_{t-1}\|.}$ Similarly, the cosine of the angle is only perturbed by the projection of ${G_t}$ onto ${U.}$ This again seems intuitive given the definition of the cosine. An important consequence is this. Initially, the cosine between ${U}$ and ${X_0}$ might be very small, e.g. ${\Omega(\sqrt{k/n})}$ if we pick ${X_0}$ randomly. Fortunately, the cosine is only affected by a ${k}$-dimensional projection of ${G_t}$ which we expect to be much smaller than the total norm of ${G_t.}$ The lemma has an intuitive geometric interpretation expressed in the following picture: It’s straightforward to prove the lemma when ${k=1}$. We simply write out the definition on the left hand side, plug in the update rule that generated ${X_t}$ and observe that the normalization terms in the numerator and denominator cancel. This leaves us with the stated expression after some simple manipulations. For larger dimension we might be troubled by the fact that we applied the Gram-Schmidt algorithm. Why would that cancel out? Interestingly it does. Here’s why. The tangent satisfies the identity: $\displaystyle \tan\theta(U,X_t) = \|(V^T X_t)(U^TX_t)^{-1}\|.$ It’s perhaps not obvious, but not more than an exercise to check this. On the other hand, ${X_t = Y_t R}$ for some invertible linear transformation ${R.}$ This is a property of orthonormalization. Namely, ${X_t}$ and ${Y_t}$ must have the same range. But ${(U^TY_tR)^{-1}=R^{-1}(U^T Y_t)^{-1}}$. Hence $\displaystyle \|(V^T X_t)(U^TX_t)^{-1}\| = \|(V^T Y_t)(U^TY_t)^{-1}\|.$ It’s now easy to proceed. First, by the submultiplicativity of the spectral norm, $\displaystyle \|(V^T Y_t)(U^TY_t)^{-1}\| \le \|(V^T Y_t)\|\cdot\|(U^TY_t)^{-1}\|.$ Consider the first term on the right hand side. We can bound it as follows: $\displaystyle \|(V^T Y_t)\|\!\le\!\|V^TAX_{t-1}\| + \|V^T G_t\|\!\le\! \sigma_{k+1}\sin\theta(U,X_{t-1}) + \|V^T G_t\|.$ Here we used that the operation ${V^T A}$ eliminates the first ${k}$ singular vectors of ${A.}$ The resulting matrix has spectral norm at most ${\sigma_{k+1}}$. The second term follows similarly. We have ${\|(U^T Y_t)^{-1}\| = 1/\sigma_{\mathrm{min}}(U^TY_t)}$ and $\displaystyle\!\!\!\sigma_{\mathrm{min}}(U^TY_{t})\!\ge\!\sigma_{\mathrm{min}}(U^TAX_{t-1})\!-\!\|U^T G_t\|\!\ge\! \sigma_k\cos\theta(U,X_{t-1})\!-\!\|U^T G_t\|\!\!\!$ This concludes the proof of the lemma. ## Applications and Pointers It’s been a long post, but it’d be a shame not to mention some applications. There are primarily three applications that sparked my interest in the robustness of the Power Method. The first application is in privacy-preserving singular vector computation. Here the goal is to compute the dominant singular vectors of a matrix without leaking too much information about individual entries of the matrix. We can formalize this constraint using the notion of Differential Privacy. To get a differentially private version of the Power Method we must add a significant amount of Gaussian noise at each step. To get tight error bounds it’s essential to distinguish between the projections of the noise term onto the space spanned by the top singular vectors and its orthogonal complement. You can see the full analysis here. Another exciting motivation comes from the area of Matrix Completion. Here, we only see a subsample of the matrix ${A}$ and we wish to compute its dominant singular vectors from this subsample. Alternating Minimization (as briefly mentioned above) is a popular and successful heuristic in this setting, but rather difficult to analyze. Jain, Netrapalli and Sanghavi observed that it can be seen and analyzed as an instance of the noisy power method that we discussed here. The error term in this case arises because we only see a subsample of the matrix and cannot evaluate exact inner products with ${A.}$ These observations lead to rigorous convergence bounds for Alternating Minimization. I recently posted some improvements. Surprisingly, there’s a fascinating connection between the privacy setting and the matrix completion setting. In both cases the fundamental parameter that controls the error rate is the coherence of the matrix ${A}$. Arguing about the coherence of various intermediate solutions in the Power Method is a non-trivial issue that arises in both settings and is beyond the scope of this post. Finally, there are very interesting tensor generalizations of the Power Method and robustness continues to be a key concern. See, for example, here and here. These tensor methods have a number of applications in algorithms and machine learning that are becoming increasingly relevant. While there has been some progress, I think it is fair to say that much remains to be understood in the tensor case. I may decide to work on this once my fear of tensor notation recedes to bearable levels. ### /r/compsci #### How much Calculus is used in a comp sci major? I am a high school senior looking to major in computer science at college. I am currently taking AB Calc (1st semester of calculus) and was wondering how much this is really need in a comp sci major. I know you have to go pretty far in math for the actual degree but how often is calculus used in the real world for programming. submitted by something1391 [link] [26 comments] ### StackOverflow #### ZeroMQ PULL service not scaling, doesn't close connections The challenge is to simply copy and PUSH several hundred http requests/sec from PHP to a single C++ PULL server. The PULL server simply writes the request to STDOUT. Under load, the system handles the load until the PULL server exhausts its open file limit of 10,000, then it stops accepting connections and pegs itself at 100% CPU usage. The meat of the server looks like this (with all the error catching code removed:) // Crank up zmq ctx = zmq_ctx_new (); // Pull socket type in the pipeline pattern s = zmq_socket (ctx, ZMQ_PULL); rc = zmq_bind (s, socketdsn); // Create a buffer to store messages char buffer[MAX_BUFFER] = {0}; while (true) { // Get the message rc = zmq_recv (s, buffer, MAX_BUFFER, 0); // Tack a null at the end of the string so printf will work... buffer[rc] = 0; // Output the string. printf("%s\n", buffer); } rc = zmq_close (s); rc = zmq_term (ctx);  The meat of the PHP PUSH server looks like this: $dsn = "tcp://localhost:5555";
// Create context
$ctx = new ZMQContext();$socket = $ctx->getSocket(ZMQ::SOCKET_PUSH, 'my socket');$socket->connect($dsn, true);$rc = $socket->send($message);
$socket->close($dsn);


My guess is that the PULL server must manually close it's connection after a message is received. The PUSH client being a PHP process, the PULL server will never see another message from it and closing the connection is safe. But there doesn't appear to be an appropriate function for that. Any ideas?

### Fefe

#### Kurze Durchsage von Goldman Sachs: Wenn die Briten ...

Kurze Durchsage von Goldman Sachs: Wenn die Briten aus der EU austreten, dann zieht Goldman Sachs von London weg, irgendwo in die Eurozone. Frankfurt oder Paris wahrscheinlich.

Diese Ansage ist die nukleare Option der Bankster. Ich hatte das im März schonmal ausgeführt. Wenn die Londoner Finanzmafia ansagt, dass England in der EU bleiben muss, dann wird England in der EU bleiben.

Update: Oh und lest mal auch den letzten Absatz, vorletzter Satz:

Auch die in Europa geplante Finanztransaktionssteuer, mit der Spekulationsgeschäfte eingedämmt werden sollen, sieht die amerikanische Investmentbank mittlerweile gelassener. „Die wird nicht zum Geschäfts-Killer werden. Wir werden eine Kompromisslösung bekommen“, erwartet Sherwood.
Entweder deren Lobbyisten haben da eine krasse Entkernung vorgenommen oder die Hölle ist gerade zugefroren. Vielleicht sowas wie die Klima-Emissionsrechte, bei der am Ende rauskam, dass in Deutschland niemand welche kaufen musste, weil der Staat genug verschenkt hat.

### Dave Winer

#### The Snowden leaks and the billionaire

A quick follow-up to Sunday's post.

There's still a question that's unresolved, and it wouldn't be important, or anyone's business really, if it weren't for the documents leaked by Edward Snowden, that were and presumably still are part of this new venture. If it was just Greenwald, Omidyar & Co -- it would certainly be interesting, but it wouldn't be so questionable (I use that term literally, no implication of wrong-doing, just saying that it's something that we have real questions about).

The main question is this: What happened?

The NSA documents still seem to reside at the Guardian. Are there now two copies, one with Greenwald and one with his (former) employer? Does he still write for the Guardian? I'm not clear about that either. What's his status at the Omidyar company? Is he a journalist-only, or is he also a principal of the company? When he speaks, is he a company spokesperson or a journalist discussing his journalistic work? Will we know when he's wearing which of these hats, if he actually has two?

Again, this wouldn't be important if he were writing about the Knicks or the weather, but he's reporting on issues of importance to people all over the world. How this is handled, the custody of the Snowden documents, and what's to become of them, is the matter of importance. It's not just important to the government and Greenwald, it's important to everyone. And it's not okay to just brush the questions aside, as they have, so far.

I can't imagine Greenwald accepting the answers his company gave to the questions we asked here. Regardless, I don't accept them. They're valid questions, and deserve a thoughtful response.

### Planet Theory

#### SIGACT NEWS column

From Jennifer Welch:

I’m honored to take over from Idit Keidar as editor of the SIGACT News Distributed Computing Column. As in the past, the December column is devoted to reporting on notable happenings in the community during the year:

- “2013 Dijkstra Prize on Distributed Computing”
– “2013 Principles of Distributed Computing Doctoral Dissertation Award”
– “PODC 2013 Review” by Nicolas Braud-Santoni
– “DISC 2013 Review” by Shahar Timnat
– “Bremen Workshop Review” by Ami Paz

An archive of the column is available online at:

https://parasol.tamu.edu/~welch/sigactNews/

Enjoy!

– Jennifer Welch

### StackOverflow

#### select first item of a collection that satisfies given predicate in clojure

Is there a function in clojure that (given a predicate and a collection), selects the first item that satisfies the given predicate and stop the iteration?

for example:

(select-first #(> % 10) (range))
=> 11


If not, maybe someone could hint me about an idiomatic implementation

### Daniel Lemire

#### If you are serious about climate change… stop attending conferences?

We know that spending energy tends to release carbon in the atmosphere. In turn this warms up the Earth in the long run. This change might be detrimental.

Being an academic, I am surrounded by people who are seriously concerned by climate change. These same academics are often insistent that we should penalize wealthy companies that produce or use fossil fuels.

However academics themselves contribute to climate change. In fact, given that academics travel more than most people, they are likely to be contribute substantially to the problem. Indeed, academics do move around quite a bit: attending meetings, going to classes, flying to conferences…

Achtena et al. have measured the carbon footprint of academics. They find that transportation accounts for 75% of the carbon footprint of a PhD student. Achtena et al. find that conference attendance alone accounts for 35% of said carbon footprint.

I have yet to meet a single researcher urging his colleagues to stop attending academic conferences. It appears that climate researchers attend as many conferences as their colleagues.

I do ask researchers who travel a lot about their carbon footprint. Invariably they dismiss my question. Sometimes, they are argue that their footprint isn’t so large. Or they point out that it is justified by the importance of their work (compared with, say, business executives).

I expect that some type of travel has been reduced. For example, when I sit on a national grant committee, I no longer expect to have to travel to the capital. But I also observe that the number of “international” academic conferences has been booming. There are more of them and some have gotten larger.

Maybe the argument is that conferences in exotic places are essential for scientific productivity. But where is the evidence? Even if Skype meetings and online exchanges are less effective, are they so bad that we absolutely need to move thousands of researchers to Las Vegas every year so that research questions can be debated?

While this is hardly ever said, a sizeable fraction of government research funding goes toward paying expensive travel… because it is a nice perk. Lots of people love getting paid travel. And some of them like to travel to conferences where they will complain about how little is done against climate change…

How is this not cognitive dissonance?

### Planet Emacsen

#### Got Emacs?: Your chance to influence Emacs development! Participate in an Emacs survey

The emacs development team is running a survey on what packages to enable by default.  To do that, they need your responses on the packages that you use.  You could parse your .emacs or do M-x report-emacs-bug and see the buffer for the list of modes enabled for you.  The survey link is below

http://www.emacswiki.org/emacs/FrequentlyEnabledPackages_Emacs244_Survey

### TheoryOverflow

#### What is higher-order in higher-order abstract syntax?

I understand that using higher-order abstract syntax essentially means using host (meta) language abstraction facilities to represent binders in embedded (object) language. But, Why exactly is it higher-order? Why is traditional abstract syntax called first-order abstract syntax?

I read the paper (Frank Pfenning, and Conal Elliott, Higher-Order Abstract Syntax, PLDI'88) that introduced HOAS as we use today. I could not quite understand the following paragraph ($\S$ 3.2) that describes encoding in HOAS:

In this first generalization, simply typed lambda terms represent programs. .. . The lexical terminals and the operators that do not introduce variable bindings are, respectively, first-order and second-order constants of the $\lambda$-calculus. Another crucial change occurs for the variables. Operators in the object language that are binding constructs are now explicitly encoded as third-order constants. As the following examples will show, this requires that bound object language variables actually become variables in the typed lambda calculus.

Particularly, I am not convinced why encodings for binding-introducing constructs like let (shown in Figure 1 of the paper) are considered third-order.

### StackOverflow

#### What is the best way to manage mutable state?

I just finished Martin Odersky's scala class at Coursera. Scala being my first FP language, I was excited by the idea of limiting mutable state. This allows for much easier concurrency and also makes the code super maintainable.

While learning all this, I realized you could guarantee the mutability of an object as long as it had no mutable variables and only referenced immutable objects. So now I can do everything by creating a new state instead of modifying an old one, use tail recursion when possible.

Great. So I can only do this so far up the chain. At some point, my application needs to be able to modify some existing state. I know where put in concurrency control at this point, locks, blah blah. I'm still defaulting to my standard multi-threaded concurrency control I've always used.

Oh scala community, is there a better way? Monads maybe?

EDIT: this question is a bit general, so I wanted to give a use case: I have a machine learning algorithm that stores several collections of data. They have functions that return updated representations of the data (training, etc), all immutable. Ultimately I can keep this return-updated-state pattern going up the chain to the actual object running the simulation. This has a state that is mutable and holds references to the collections. I may want to distributed to multi-cores, or multi-system.

#### Scala: Mock companion object & Model vs Service approach

I have seen multiple questions regarding as how to mock a companion object with frequently mentioned answer being to use scalamock but according to the doc for version 3, mocking companion objects is still marked for future.

So what is the way to mock a companion object? If not is there a better way to do things than what I already have to enable testing?

Code using play framework:

Model

case class Article(id: String, preview: String)

object Article {
def get(id: String) = {
......
}
}


ArticleResource

class ArticleResource extends Controller {
def getArticle(id: String) = authenticate {
......
Article.get(id)
......
}
}


How do I mock Article.get(id)?

Before when I had it working, I had structured the code differently:

Approach for working tests

Model

case class Article(id: String, preview: String)


Service

class ArticleService {
def get(id: String) = {
......
}
}

object ArticleService {
def apply = new ArticleService
}


ArticleResource

class ArticleResource(articleService: ArticleService) {
def getArticle(id: String) = authenticate {
......
articleService.get(id)
......
}
}

object ArticleResource extends controllers.ArticleResource(ArticleService())


In the test I injected a mock[ArticleService] into the Resource for testing.

I kept hearing the former approach (models) is the "right" approach, I never understood why so I tried it and ran into problems into testing and still I do not understand why it is the better approach. (I am actually feeling more strongly towards the latter approach now considering the difficulties with testing)

So I actually ended up asking 2 questions:

1. Is there a way to mock companion object?
2. Is there a "right" approach as how code is structured and if so why?

### Matthew Green

#### How does the NSA break SSL?

 (source)
A few weeks ago I wrote a long post about the NSA's 'BULLRUN' project to subvert modern encryption standards. I had intended to come back to this at some point, since I didn't have time to discuss the issues in detail. But then things got in the way. A lot of things, actually. Some of which I hope to write about in the near future.

But before I get there, and at the risk of boring you all to tears, I wanted to come back to this subject at least one more time, if only to pontificate a bit about a question that's been bugging me.

You see, the NSA BULLRUN briefing sheet mentions that NSA has been breaking quite a few encryption technologies, some of which are more interesting than others. One of those technologies is particularly surprising to me, since I just can't figure how NSA might be doing it. In this extremely long post I'm going to try to dig a bit deeper into the most important question facing the Internet today.

Specifically: how the hell is NSA breaking SSL?
 Section of the BULLRUN briefing sheet. Source: New York Times.
To keep things on target I'm going to make a few basic ground rules.

First, I'm well aware that NSA can install malware on your computer and pwn any cryptography you choose. That doesn't interest me at all, for the simple reason that it doesn't scale well. NSA can do this to you, but they can't do it for an entire population. And that's really what concerns me about the recent leaks: the possibility that NSA is breaking encryption for the purposes of mass surveillance.

For the same reason, we're not going to worry about man-in-the-middle (MITM) attacks. While we know that NSA does run these, they're also a very targeted attack. Not only are MITMs detectable if you do them at large scale, they don't comport with what we know about how NSA does large-scale interception -- mostly via beam splitters and taps. In other words: we're really concerned about passive surveillance.

The rules above aren't absolute, of course. We will consider limited targeted attacks on servers, provided they later permit passive decryption of large amounts of traffic; e.g., decryption of traffic to major websites. We will also consider arbitrary modifications to software and hardware -- something we know NSA is already doing.

One last point: to keep things from going off the rails, I've helpfully divided this post into two sections. The first will cover attacks that use only known techniques. Everything in this section can be implemented by a TAO employee with enough gumption and access to software. The second section, which I've titled the 'Tinfoil Hat Spectrum' covers the fun and speculative stuff -- ranging from new side channel attacks all the way to that huge quantum computer the NSA keeps next to BWI.

Attacks that use Known Techniques

Theft of RSA keys. The most obvious way to 'crack' SSL doesn't really involve cracking anything. Why waste time and money on cryptanalysis when you can just steal the keys? This issue is of particular concern in servers configured for the TLS RSA handshake, where a single 128-byte server key is all you need to decrypt every past and future connection made from the device.

In fact, this technique is so obvious that it's hard to imagine NSA spending a lot of resources on sophisticated cryptanalytic attacks. We know that GCHQ and NSA are perfectly comfortable suborning even US providers overseas. And inside our borders, they've demonstrated a willingness to obtain TLS/SSL keys using subpoena powers and gag orders. If you're using an RSA connection to a major website, it may be sensible to assume the key is already known.

Of course, even where NSA doesn't resort to direct measures, there's always the possibility of obtaining keys via a remote software exploit. The beauty is that these attacks don't even require remote code execution. Given the right vulnerability, it may simply require a handful of malformed SSL requests to map the full contents of the OpenSSL/SChannel heap.

 Source: New York Times
Suborning hardware encryption chips. A significant fraction of SSL traffic on the Internet is produced by hardware devices such as SSL terminators and VPN-enabled routers. Fortunately we don't have to
speculate about the security of these devices -- we already know NSA/GCHQ have been collaborating with hardware manufacturers to 'enable' decryption on several major VPN encryption chips.

The NSA documents aren't clear on how this capability works, or if it even involves SSL. If it does, the obvious guess is that each chip encrypts and exflitrates bits of the session key via 'random' fields such as IVs and handshake nonces. Indeed, this is relatively easy to implement on an opaque hardware device. The interesting question is how one ensures these backdoors can only be exploited by NSA -- and not by rival intelligence agencies. (Some thoughts on that here.)

Side channel attacks. Traditionally when we analyze cryptographic algorithms we concern ourselves with the expected inputs and outputs of the system. But real systems leak all kinds of extra information. These 'side channels' -- which include operation time, resource consumption, cache timing, and RF emissions -- can often be used to extract secret key material.

The good news is that most of these channels are only exploitable when the attacker is in physical proximity to a TLS server. The bad news is that there are conditions in which the attacker can get close. The most obvious example involves virtualized TLS servers in the cloud setting, where a clever attacker may share physical resources with the target device.

A second class of attack uses remote timing information to slowly recover an RSA key. These attacks can be disabled via countermeasures such as RSA blinding, though amusingly, some 'secure' hardware co-processors may actually turn these countermeasures off by default! At very least, this makes the hardware vulnerable to attacks by a local user, and could even facilitate remote recovery of RSA keys.

Weak random number generators. Even if you're using strong Perfect Forward Secrecy ciphersuites, the security of TLS depends fundamentally on the availability of unpredictable random numbers. Not coincidentally, tampering with random number generator standards appears to have been a particular focus of NSA's efforts.

Random numbers are critical to a number of elements in TLS, but they're particularly important in three places:
1. On the client side, during the RSA handshake. The RNG is used to generate the RSA pre-master secret and encryption padding. If the attacker can predict the output of this generator, she can subsequently decrypt the entire session. Ironically, a failure of the server RNG is much less devastating to the RSA handshake.*
2. On the client or server side, during the Diffie-Hellman handshake(s). Since Diffie-Hellman requires a contribution from each side of the connection, a predictable RNG on either side renders the session completely transparent.
3. During long-term key generation, particularly of RSA keys. If this happens, you're screwed.
And you just don't need to be that sophisticated to weaken a random number generator. These generators are already surprisingly fragile, and it's awfully difficult to detect when one is broken. Debian's maintainers made this point beautifully back in 2008 when an errant code cleanup reduced the effective entropy of OpenSSL to just 16 bits. In fact, RNGs are so vulnerable that the challenge here is not weakening the RNG -- any idiot with a keyboard can do that -- it's doing so without making the implementation trivially vulnerable to everyone else.

The good news is that it's relatively easy to tamper with an SSL implementation to make it encrypt and exfiltrate the current RNG seed. This still requires someone to physically alter the library, or install a persistent exploit, but it can be done cleverly without even adding much new code to the existing OpenSSL code. (OpenSSL's love of function pointers makes it particularly easy to tamper with this stuff.)

If tampering isn't your style, why not put the backdoor in plain sight? That's the approach NSA took with the Dual_EC RNG, standardized by NIST in Special Publication 800-90. There's compelling evidence that NSA deliberately engineered this generator with a backdoor -- one that allows them to break any TLS/SSL connection made using it. Since the generator is (was) the default in RSA's BSAFE library, you should expect every TLS connection made using that software to be potentially compromised.

And I haven't even mentioned Intel's plans to replace the Linux kernel RNG with its own hardware RNG.

Esoteric Weaknesses in PFS systems. Many web servers, including Google and Facebook, now use Perfect Forward Secrecy ciphersuites like ephemeral Diffie-Hellman (DHE and ECDHE). In theory these ciphersuites provide the best of all possible worlds: keys persist for one session and then disappear once the connection is over. While this doesn't save you from RNG issues, it does make key theft a whole lot more difficult.

PFS ciphersuites are a good thing, but a variety of subtle issues can cramp their style. For one thing, the session resumption mechanism can be finicky: session keys must either be stored locally, or encrypted and given out to users in the form of session tickets. Unfortunately, the use of session tickets somewhat diminishes the 'perfectness' of PFS systems, since the keys used for encrypting the tickets now represent a major weakness in the system. Moreover, you can't even keep them internal to one server, since they have to be shared among all of a site's front-end servers! In short, they seem like kind of a nightmare.

A final area of concern is the validation of Diffie-Hellman parameters. The current SSL design assumes that DH groups are always honestly generated by the server. But a malicious implementation can violate this assumption and use bad parameters, which enable third party eavesdropping. This seems like a pretty unlikely avenue for enabling surveillance, but it goes to show how delicate these systems are.

The Tinfoil Hat Spectrum

I'm going to refer to the next batch of attacks as 'tinfoil hat' vulnerabilities. Where the previous issues all leverage well known techniques, each of the following proposals require totally new cryptanalytic techniques. All of which is a way of saying that the following section is pure speculation. It's fun to speculate, of course. But it requires us to assume facts not in evidence. Moreover, we have to be a bit careful about where we stop.

So from here on out we are essentially conducting a thought-experiment. Let's imagine the NSA has a passive SSL-breaking capability; and furthermore, that it doesn't rely on the tricks of the previous section. What's left?

The following list begins with the most 'likely' theories and works towards the truly insane.

Breaking RSA keys. There's a persistent rumor in our field that NSA is cracking 1024-bit RSA keys. It's doubtful this rumor stems from any real knowledge of NSA operations. More likely it's driven by the fact that cracking 1024-bit keys is highly feasible for an organization with NSA's resources.

How feasible? Several credible researchers have attempted to answer this question, and it turns out that the cost is lower than you think. Way back in 2003, Shamir and Tromer estimated $10 million for a purpose-built machine that could factor one 1024-bit key per year. In 2013, Tromer reduced those numbers to about$1 million, factoring in hardware advances. And it could be significantly lower. This is pocket change for NSA.

Along similar lines, Bernstein, Heninger and Lange examined at the feasibility of cracking RSA using distributed networks of standard PCs. Their results are pretty disturbing: in principal, a cluster about the size of the real-life Conficker botnet could do serious violence to 1024-bit keys.

Given all this, you might ask why this possibility is even in the 'tinfoil hat' category. The simple answer is: because nobody's actually done it. That means it's at least conceivable that the estimates above are dramatically too high -- or even too low. Moreover, RSA-1024 keys are being rapidly being phased out. Cracking 2048 bit keys would require significant mathematical advances, taking us much deeper into the tinfoil hat.**

Cracking RC4. On paper, TLS supports a variety of strong encryption algorithms. In practice, about half of all TLS traffic is secured with the creaky old RC4 cipher. And this should worry you -- because RC4 is starting to show its age. In fact, as used in TLS it's already vulnerable to (borderline) practical attacks. Thus it seems like a nice candidate for a true cryptanalytic advance on NSA's part.

Unfortunately the problem with this theory is that we simply don't know of any attack that would allow the NSA to usefully crack RC4! The known techniques require an attacker to collect thousands or millions of ciphertexts that are either (a) encrypted with related keys (as in WEP) or (b) contain the same plaintext. The best known attack against TLS takes the latter form -- it requires the victim to establish billions of sessions, and even then it only recovers fixed plaintext elements like cookies or passwords.

The counterargument is that the public research community hasn't been thinking very hard about RC4 for the past decade -- in part because we thought it was so broken people had stopped using it (oops!) If we'd been focusing all our attention on it (or better, the NSA's attention), who knows what we'd have today.

If you told me the NSA had one truly new cryptanalytic capability, I'd agree with Jake and point the finger at RC4. Mostly because the alternatives are far scarier.

New side-channel attacks. For the most part, remote timing attacks appear to have been killed off by the implementation of countermeasures such as RSA blinding, which confound timing by multiplying a random blinding factor into each ciphertext prior to decryption. In theory this should make timing information essentially worthless. In practice, many TLS implementations implement compromises in the blinding code that might resurrect these attacks, things like squaring a blinding factor between decryption operations, rather than generating a new one each time. It's quite unlikely there are attacks here, but who knows.

Goofy stuff. Maybe NSA does have something truly amazing up its sleeve. The problem with opening this Pandora's box is that it's really hard to get it closed again. Did Jerry Solinas really cook the NIST P-curves to support some amazing new attack (which NSA knew about way back in the late 1990s, but we have not yet discovered)? Does the NSA have a giant supercomputer named TRANSLTR that can brute-force any cryptosystem? Is there a giant quantum computer at the BWI Friendship annex? For answers to these questions you may as well just shake the Magic 8-Ball, cause I don't have a clue.

Conclusion

We don't know and can't know the answer to these things, and honestly it'll make you crazy if you start thinking about it. All we can really do is take NSA/GCHQ at their word when they tell us that these capabilities are 'extremely fragile'. That should at least give us hope.

The question now is if we can guess well enough to turn that fragility from a warning into a promise.

Notes:

* A failure of the server RNG could result in some predictable values like the ServerRandom and session IDs. An attacker who can predict these values may be able to run active attacks against the protocol, but -- in the RSA ciphersuite, at least -- they don't admit passive compromise.

** Even though 1024-bit RSA keys are being eliminated, many servers still use 1024-bit for Diffie-Hellman (mostly for efficiency reasons). The attacks on these keys are similar to the ones used against RSA -- however, the major difference is that fresh Diffie-Hellman 'ephemeral' keys are generated for each new connection. Breaking large amounts of traffic seems quite costly.

### QuantOverflow

#### How accurately can the LIBOR market model price a floating note

I am considering some hedging strategy where portfolios of derivatives are built so that each portfolio is equivalent to a floating note, even if the instruments in the portfolio might be quite complicated, e.g. bermudean snowballs.

I plan to price numerically these portfolios in a forward LIBOR market model calibrated to swaption prices and I am wondering which accuracy I should expect on the total portfolio prices, i.e. on the pricing of floater notes.

### StackOverflow

#### Either Monad and Exceptional Cases

If I have a function returning an Either such as GetUserFromDb(int id) and the database is offline should I catch the error in the function and wrap it in a failure / Left case or should I let it bubble out as an exception as there is nothing I can do and it really is an exceptional situation.

Cheers,

Blair

#### Testing static routes in compojure

I am trying to write a test for the compojure static content route. I am testing the routes by examining the ring response directly.

A minimal working example is as follows:

;; src/testing-webapps.core.clj
(ns testing-webapps.core
(:use [compojure.core]
[compojure.route   :as route]))

(defroutes web-app
(route/resources "/")
(route/not-found "404"))

;; test/testing-webapps.core_test.clj
(ns testing-webapps.core-test
(:require [clojure.test :refer :all]
[testing-webapps.core :refer :all]))

(defn request [resource web-app & params]
(web-app {:request-method :get :uri resource :params (first params)}))

(deftest test-routes
(is (= 404 (:status (request "/flubber" web-app))))
(is (= "404" (:body (request "/flubber" web-app))))
(is (= 200 (:status (request "/test.txt" web-app)))))


Testing the 404 route works fine but calling (request "/test.txt" web-app) leads to an unexpected NullPointerException in ring.middleware.file-info/not-modified-since?.

Here is the top part of the stacktrace:

ERROR in (test-routes) (file_info.clj:27)
Uncaught exception, not in assertion.
expected: nil
actual: java.lang.NullPointerException: null
at ring.middleware.file_info$not_modified_since_QMARK_.invoke (file_info.clj:27) ring.middleware.file_info$file_info_response.doInvoke (file_info.clj:44)
clojure.lang.RestFn.invoke (RestFn.java:442)
ring.middleware.file_info$wrap_file_info$fn__917.invoke (file_info.clj:64)
[...]


The static route works fine in the browser but not when called through my request function.

Is there an easier way to test a static route in compojure and why do I get a NullPointerException when calling the static route with my own request map?

#### Compute the inverse of function

I want to compute and plot the inverse of given function f. I have the following code in R :

############## Parameters ######
r1 <- 0.0125
r2 <- 0.0305
S1 <- 0.0400
S2 <- 0.0900
s1 <- sqrt(S1)
s2 <- sqrt(S2)

rho <- -0.45
############## function f ###########
f <- function(u) ((((u-r2)/(r1-r2))^2)*((s1)^2)) +
(((1-((u-r2)/(r1-r2)))^2)*((s2)^2)) +
(2*(((u-r2)/(r1-r2)))*(1-((u-r2)/(r1-r2)))*rho*s1*s2)

f(r1)

#### the values is equal to 0.04


The problem is how to invert this function, I try to use the function uniroot but I was not able to solve my problem. this was the code that I used :

########  First I define the function inverse :

inverse = function (f, lower = 0, upper = 0.035) { function (y)  uniroot((function (x) f(x) - y), lower = lower, upper = upper)[1]}

######## second I define the inverse of f using inverse

f_inverse = inverse(function (u) f(u),  0.0, 0.035)


#### How can I build a new collection which consists of elements that satisfy some condition?

If I have a list like this in scala:

val list = List(
Map("val1" -> 1, "val2" -> 2),
Map("val1" -> 3, "val2" -> 4),
Map("val1" -> 5, "val2" -> 6),
Map("val1" -> 7, "val2" -> 8)
)


And I like to create another list where elements match certain condition like this:

val newList = list map { el /*match (el("val1") < 5) here*/ =>
el /*if condition is met, add element to new list*/
}


Then result would be something like this:

List(
Map("val1" -> 1, "val2" -> 2),
Map("val1" -> 3, "val2" -> 4)
)


Is something like this possible and if so then how? I'd like to make this work from functional programming perspective.

#### Translate mongodb aggregate function to ReactiveMongo and scala

I'm trying to implement an aggregate method in ReactiveMongo but I'm a bit stuck.

I have the following dataset:

{
"_id" : ObjectId("522891aa40ef0b5d11cb9232"),
"created" : 1378390442167,
"origin" : 2,
"rating" : 3,
"status" : 4,
"text" : "",
"updated" : 1378563426223,
"userInfo" : {
"firstName" : "Person",
"lastName" : "Person",
"email" : "person@person.com",
"fbPublish" : false
},
"venueInfo" : {
"isAgent" : false,
"name" : "Company",
"id" : 1234
}
},
{
"_id" : ObjectId("522891aa40ef0b5d11cb9233"),
"created" : 1378390442167,
"origin" : 2,
"rating" : 3,
"status" : 4,
"text" : "",
"updated" : 1378563426223,
"userInfo" : {
"firstName" : "Person2",
"lastName" : "Person2",
"email" : "person2@person.com",
"fbPublish" : false
},
"venueInfo" : {
"isAgent" : false,
"name" : "Company2",
"id" : 4321
}
},
{
"_id" : ObjectId("522891aa40ef0b5d11cb9234"),
"created" : 1378390442167,
"origin" : 2,
"rating" : 3,
"status" : 4,
"text" : "",
"updated" : 1378563426223,
"userInfo" : {
"firstName" : "Person3",
"lastName" : "Person3",
"email" : "person3@person.com",
"fbPublish" : false
},
"venueInfo" : {
"isAgent" : false,
"name" : "Company",
"id" : 1234
}
}


The following aggregate function:

db.reviews.aggregate(
{$match:{status:{"$ne":1}}},
{$group: { _id: "$venueInfo.id", total:{"$sum":1}}} )  gives me: { "result" : [ { "_id" : 1234, "total" : 2 }, { "_id" : 4321, "total" : 1 } ] }  I tried to implement this in ReactiveMongo: def aggregate() = { val command = Aggregate(collection.name, Seq( GroupField("venueInfo.id")("total" -> SumValue(1)), Match(BSONDocument("status" -> 1)) )) val result = collection.db.command(command) result.map { value => { println(s"got value$value")
}

}


And that gives me:

got value Stream(BSONDocument(<non-empty>), ?)


As you can see I get a Stream back. So my question is: How do I handle this stream in a correct way so that I can work with the values and show them later in the view?

### TheoryOverflow

#### Examples of context-free languages with a non-context-free complements

Context-free languages are not closed under complementation. In the lectures we have been given the same argument as here on Wikipedia. But this only shows that one of the three languages $A$, $B$, and $\overline A \cup \overline B$ is a context-free language with a non-context-free complement, not for which one of these this is true. So what is it?

Also, is there a minimal and elegant example of a context-free language witch a non-context-free complement, maybe over a binary alphabet?

### StackOverflow

#### scala-hector-cassandra-cli: how to delete all colums of column Family by rowkeyId? [closed]

userId---->RowKey,
userData--->column family name,
dataId--->column name


but i have also some dynamic cloumn names, and i dont know the names of those columns.
is there any way to delete the complete row.

def deleteData(dataId:String) {
userId=session.get("userId").get
val mutator1 = HFactory.createMutator(keyspace, se)
val query1 = HFactory.createStringColumnQuery(keyspace)
mutator1.delete(userId, "userData", dataId, se)
} //end removeData


#### SBT Multi-Project Build with dynamic external projects?

Let's say we have an SBT project bar with a dependency on some artifact foo:

val bar = Project('bar', file('.')).settings(
libraryDependencies += "com.foo" % "foo" % "1.0.0"
)


However, in certain cases, I want to checkout the source of foo and have SBT load the source from my file system instead of the published artifact; that way, I could make local changes to foo and immediately test them with bar without having to publish anything.

val foo = Project('foo', file('foo'))

val bar = Project('bar', file('.')).dependsOn(foo)


We have a spec.json file in the root folder of bar that already specifies if foo should be used from source or as an artifact. Is there any way to setup my build to read this file and add dependsOn or libraryDependencies based on the value in spec.json? '

It's easy enough to do this for libraryDependencies:

val bar = Project('bar', file('.')).settings(
libraryDependencies ++=
if (containsFoo(baseDirectory.value / "spec.json")) {
Seq()
} else {
Seq("com.foo" % "foo" % "1.0.0")
}
)


However, we can't find any way to set do anything "dynamic" in dependsOn, such as reading the baseDirectory SettingKey.

#### TDD scala tutorial

I am working with scala for couple on months. I want to focus on functional programming using scala . I am looking for a book that shows real world application using scala . and preferred (if there is one) with TDD approach
books that I read are concept and syntax , now I want to dive in FP programming using scala with TDD approach and looking for a good tutorial using real world applications as a reference . any recommendations ?

### Planet Emacsen

#### Irreal: Working with Images in Emacs

Over at Bytopia, Alex has a beautiful post on using Emacs to work with images. Alex is trying to train an OCR classifier to recognize characters from grocery store receipts. He started by scanning 4 or 5 receipts and breaking the characters into individual images. Next he needed to assign the proper character to each image.

That’s where Emacs comes in. First, he uses turn-on-iimage-mode to turn a buffer of a directory of image files into a buffer of the corresponding images. A nice thing about this mode is that when the cursor is on an image you can still read the image file name. That and a bit of Elisp allow Alex to specify the character in each image (or delete the image if it’s garbled) and prepend the character to the file name. With a little more massaging, these images are used to train the classifier.

It’s astounding how little code all this takes. Head on over to Bytopia to see how he does it. Once again, Emacs and just a little Elisp easily solves a problem that would take considerable effort if one where to do it with a script or, worse, as a C application.

### StackOverflow

#### ZeroMQ and local FIFO

I have two processes ('sender' and 'receiver') that need to communicate via a transient one way FIFO communication pipe, locally on a single machine. Here's what I would like to happen (using language that's closer to Unix domain sockets):

• Sender "creates" pipe at well-known address, and immediately sends messages down it
• At some point (before or after sender "creates" pipe), receiver connects to pipe
• Sender "closes" pipe
• Reader notices that all messages have been read (possibly that pipe is closed)

My question is: how do I implement this with ZeroMQ? "PUB/SUB", "PUSH/PULL"? What is the mechanism to detect the "end of data" in a ZMQ socket? Is it possible to allow both of the orderings of the first two items above: That is whether sender or receiver attempts to connect first? If so, how?

Thanks.

#### What are the differences between Scala middleware choices? [closed]

Note: Unfortunately this question was closed, but I'm trying to maintain it for if someone else comes along with the same question.

I've been looking for a good solution to developing a service in Scala that will sit between mobile devices and existing web services.

The current list of viable options are:

There are probably more options out there. How does one decide which one to use? What are the traits (excuse the pun ;-) of a good Scala middleware choice. On the one side, I would like to go for Akka, because it is part of the TypeSafe Scala stack, but on the other, something like Finagle has a rich set of libraries and makes plumbing so easy. Spray looks nice and simple to use.

Any advice, insights or experience would be greatly appreciated. I'm sure someone out there must have some experience with some of these that they won't mind sharing.

UPDATE:

I would love for this question to be reopened. A good answer to this question will help new Scalateers to avoid related pitfalls.

UPDATE 2:

These are my own experiences since asking this question:

Finagle - I used Finagle for a project and it's rock solid.

Spray - In my latest project I'm using Spray and I'm extremely happy. The latest releases are built on Akka 2 and you can run it directly with the Spray-can library which removes the need for a web server. Spray is a set of libraries, rather than a framework and is very modular. The Learn about Spray: REST on Akka video gives a great overview, and this blog at Cakesolutions shows a really nice development approach and architecture.

UPDATE 3:

Life moves pretty fast. If you don't stop and look around once in a while, you could miss it. - Ferris Bueller

These days the choice has become simpler. In my humble opinion Spray has won the battle. It is being integrated into Akka to become the next Akka HTTP. I have been using Spray now on multiple projects and can honestly say that it's fantastic and best supported software I have ever encountered.

This does not answer the initial question, but at least gives some indication on why Spray seems like the best choice in most cases. It is extremely flexible, non-blocking and very stable. It has both client-side and server-side libraries and a great testkit. Also, have a look at these stats to get an idea on performance: Web Framework Benchmarks

### CompsciOverflow

#### Does a collision oracle for the pigeonhole subset sum problem produce solutions?

I am reading "Efficient Cryptographic Schemes Provably as Secure as Subset Sum" by R. Impagliazzo and M. Naor (paper) and came across the following statement in the proof of Theorem 3.1 (pages 10-11):

Let $\ l(n) = (1-c)n \$ for $\ c > 0 \$ ...

Given $a_1, a_2, \cdots, a_n \in \{0,1\}^{l(n)}$ and a target sum $T$, we construct an input to the collision finding algorithm as follows:

1. Let the collision finding algorithm select a (non-empty) $s_1 \in \{0,1\}^n$

2. compute $T' = \sum_{i \in s_1} a_i$. Choose a random $j$ such that $j \in s_1$ and define $a_j' = a_j - T' + T$.

3. Give the instance $a_1, a_2, \cdots , a_j', \cdots, a_n$ and $s_1$ to the algorithm that finds collisions. The algorithm attempts to find $s_2$ such that $f_{(a_1, a_2, \cdots, a_j', \cdots, a_n)}(s_2) = T'$.

If the algorithm returns $s_2$ that collides with $s_1$ and $j \notin s_2$, then $s_2$ is a solution to our original problem, since swapping $a_j$ and $a_j'$ does not affect the sum over $s_2$.

Where the emphasis is mine.

Where $f$ concatenates $\stackrel{\rightarrow}{a}$ with the sum of the $a_i$'s:

$$f( \stackrel{\rightarrow}{ a } , S) = f_{(a_1, a_2, \cdots, a_n)}(S) = \ \stackrel{\rightarrow}{a}, \sum_{i \in S} a_i \mod 2^{l(n)}$$

(taken from the top of page 3 from the same paper).

For the life of me, I don't understand how $s_2$ is a solution to the original instance. Can someone elaborate on what they mean? What am I missing?

The above definition for the subset sum problem is, if I'm not mistaken, just another form of the pigeonhole subset sum problem (i.e. $\sum_j a_j < 2^n -1$ ). If I read the above right, they are claiming that, given an oracle that finds collisions, they can then construct a solution to the original (pigeonhole) subset sum problem but I do not see how this is done. Any help would be appreciated.

### StackOverflow

#### What is the most useful feature of Scala which could allow it to take Java's place? [on hold]

I believe the most useful feature of Scala is the hierarchical access modifier scoping mechanism. I believe this is really useful as it enhances our ability to encapsulate components and thus reduce bugs, as we are better able to restrict the available operations at each point in execution. Most systems are naturally hierarchical with a higher level component controlling many lower level components, and Scala allows access to these components to be controlled in a granular way using a quantified version of the private key word. I think C# has a similar mechanism...

Could this be the single most important feature of Scala that gives it the edge over Java?

### StackOverflow

#### clojure/lein: How do I include a java file in my project?

I have a java file with a single class and I want to include it in my lein project. Where do I put it and how do I import it? (I tried putting it in the src directory under the package path but it tells me ClassNotFound)

So the java file has this package declaration:

package com.thebuzzmedia.imgscalr;


and has this class:

public class Scalr {


I put it in ~/src/com/thebuzzmedia/imgscalr/Scalr.java and tried to import it from the repl thusly:

(import '(com.thebuzzmedia.imgscalr Scalr))


And I get this:

com.thebuzzmedia.imgscalr.Scalr
[Thrown class java.lang.ClassNotFoundException]


What am I missing?

### TheoryOverflow

#### Foundational textbook(s) for Complexity and Computability on Real Numbers

It would be extremely helpful if someone can suggest foundational textbooks on Recursive Analysis (Computability over Reals) which explains connections between Computability and the Topological properties of continuous sets. I could not find any, except "Recursive Analysis" by R.L. Goodstein but I dont have any feedback on that book. I would also like to ask for suggestions regarding a book on Complexity over Real Numbers which again deals with the connection between the complexity and the topology of a set. The only book I found on the net was "Complexity and Real Computation" by Lenore Blum et al but I am not sure if that is the book which I am looking for. Thanks in advance.

### StackOverflow

#### Consumer for multiple publishers with different RabbitMQ servers in python

Hi we have a situation when several consumers are consuming messages from several different publishers, each on its own machine. Consumers are dulpicating each others. We need to be able to deal with consumer disconnect. It should receive offline messages from all publishers.

One way of doing that is with RabbitMQ. So the message queue should be in each publisher machine. And so the RabbitMQ server should be in each publisher machine also. But how to consume messages from different publishers, when each queue is on different server? We should have a several connections, one for each publisher (RabbitMQ server). We could do this with multithreading, probably, giving each blocking io loop its own thread. But is this the way it should be done?

Another option is to use zeroMQ, but we would have to write our own reliability layer, with persistent queue, acknowledgement etc. It seems to me, that using RabbitMQ is more easy.

How you think we should handle this problem?

### Fred Wilson

#### The Goodlatte Patent Bill

Yesterday the White House came out in favor of Rep Bob Goodlatte's patent bill. This is a good thing.

The Goodlatte bill doesn't have everything in it that we would like to see in a bill aimed at reducing the pain that patent trolls inflict on the innovation economy, but it is a good start and I think it can get improved in conference with the Senate.

One thing that was taken out of the Goodlatte bill that we would like to see put back in during the conference with the Senate is the broadening of the covered business method patent review process to include all business method patents.

In the five plus years that we have been working to educate government officials about the blight that patent trolls have wreaked on the startup sector, I have seen a huge shift. We have gone from elected officials being ignorant about this issue, to be aware of it, to now being as outraged as we are about the troll issue. That's a good thing and the result, I think, will be better laws and better processes to out bad patents and bad actors in the system. This is long overdue but welcome nonetheless.

### UnixOverflow

#### How to apply updates on OpenBSD, NetBSD, and FreeBSD?

I'm using OpenBSD for quite a while now. All I do, however is go from one release to the next, always just doing an update. I configured the system so it works as my router and firewall, and it works quite good like that. But, I never update packages. All I do, I just move on to the next release.

Coming from the Linux world, I'm used to apply updates a few times a week, but how do I do that on *BSD? - Or is this not part of the *BSD philosophy?

### Fefe

#### Statistik des Tages:Hassan Rouhani, Iran’s president, ...

Statistik des Tages:
Hassan Rouhani, Iran’s president, has more cabinet members with Ph.D. degrees from U.S. universities than Barack Obama does.

#### US-Vizepräsident Joe Biden ist in China zu Besuch ...

US-Vizepräsident Joe Biden ist in China zu Besuch und hat dabei gleich mit einem denkwürdigen Klogriff losgelegt:
Thanking a group of mostly young people for wanting to visit the U.S., Biden said he hoped they would learn during their visit that "innovation can only occur where you can breathe free."

"Children in America are rewarded - not punished - for challenging the status quo," Biden said. "The only way you make something totally new is to break the mold of what was old."

Ja, das konnte man ja gerade in diesem Jahr besonders gut sehen, wie die USA mit aufmüpfigen Status-Quo-Hinterfragern wie Manning und Snowden umgeht.

Schon toll, dieser Honecker-Effekt. Je weiter oben in der Hierarchie, desto weniger Realitätsbezug.

### StackOverflow

#### how to invoke a setter method on an object (that coerces types)

I wish to write a function that interops with java objects

(assoc-obj <object> "<Prop1>" <Val1> "<Prop2>" <Val2>)


which is the same as

(doto <object>
(.set<Prop1> <Val1>)
(.set<Prop2> <Val2>))


I'm working with reflection:

(let [method (->>
(seq (.getMethods java.util.Date))
(filter #(= "setDate" (.getName %)))
first)
arr (object-array 1)
_   (aset arr 0 (int 1))
d   (java.util.Date.)]
(.invoke method d arr)
d)


but I am finding that there are problems with type coercing.

Are there better/more clojurish ways of doing this?

### CompsciOverflow

#### Clarification on Tabu Search

I need some help in understanding the 'Tabu Search' Algorithm. (Wikipedia)

I miss a simple explanation to Tabu Search. Anyway, I'm trying to refer to available resources and build an understanding.

This is what I'm trying to 'digest':

• Tabu Search is an improvement over the Hill Climbing algorithm (Ref-1).

• The problem with Hill Climbing is that it does not guarantee about reaching the global optimum, because it only searches on a subset of the whole solution space. It will find the local optimum.

• To get rid of this issue, Tabu Search maintains a 'Tabu List' of previously visited states that cannot be revisited (Ref-2).
• If the tabu list is too large, the oldest candidate solution is removed and it’s no longer tabu to reconsider (Ref-3).

My questions are,

1. How does Tabu Search cure the problem of getting stuck in a local optimum? Does it increase the search-space?

2. What is the need of maintaining a list (i.e. Tabu List)? Why not just remember the optimum solution found so far?

3. When the Tabu List is too large, the oldest candidate will be removed. What if this oldest candidate is the global optimum?

If anyone could explain Tabu Search algorithm using an example, I'm sure these questions would be automatically answered.

References:

• (Ref-1) Hill Climbing, Wikipedia Article (link)

• (Ref-2) Russell, Stuart Jonathan, et al. Artificial intelligence: a modern approach. Vol. 74. Englewood Cliffs: Prentice hall, 1995. (WorldCat)

• (Ref-3) Luke, Sean. "Essentials of Metaheuristics.". (pdf)

### StackOverflow

#### Filtering elements of one list by looking at boolean values from the second list

I have two lists of equal length. I want to filter the elements of the first list by looking, if the element, with the same index in the second list, has a true boolean value.

Example:

[1,2,3,4,5]:int list

[true,false,false,true,false]:bool list

Expected result: [1,4]


I know two ways I could achieve this:

1) Write a function that takes two lists. For every element in the first list, that I want to append, check if the current(head) element of the second list is true.

2) Zip the two lists and filter it according to the boolean value.

### CompsciOverflow

#### Proving Context free

Can anybody please help me in proving a^n b^2n where n>=0 is context free language using pumping lemma. http://www.idt.mdh.se/kurser/cd5560/10_11/examination/KOMPENDIER/ContextFree/Comp-CFL-10.pdf at page 24 there is a npda that accepts this language but i am not able to prove it using Pumping lemma

### CompsciOverflow

#### Regular Grammar and Regular Language

My doubt is based on the relation between regular grammar and regular languges

$S \to Sa\ |\ a$ produces the regular language $a^+$ but this grammar is not type-3 grammar of Chomsky classification. So am I correct in stating that this is not a regular grammar even when it produces regular language?

### CompsciOverflow

#### sorting stone Problem

My friend asked this problem recently & am not sure which sorting to use for this kind of problem:-

There are 20 stones of different heights. Each stone is so heavy, we need to sort the stones such that smallest stones are to left and highest to right. As stones are heavy make algorithm which moves stone least distance possible

Which sorting method suits this kind of problem?

### StackOverflow

#### How to implement a For loop in Clojure

I'd like to implement this little code in Clojure, but I am struggling:

struct mystruct{
int id;
int price;
};

mystruct mydata[10];

for (int i=0; i<10; i++)
{
myfunction(mydata[i].id, mydata[i].price);
//other things...
}


I am a beginner with Clojure and it's really complicate for me to do something simple like this, but I am really trying to learn as much as possible as I know that there are great advantages with Clojure such using refs...

I would really appreciate if somebody could help me. Thanks!!

#### how to stop sbt package command from appending hash to the name of the jar

I cloned https://github.com/sbt/sbt-native-packager and then from sbt console ran "package"

How do I stop it from generating the hash/sha? or How do I publish it without the hash?

#### How to load configuration files for apps bundled using sbt-native-packager during runtime

I built a universal tarball using sbt-native-packager. How do I load configuration files like c3p0 and application.conf from /etc/myapp or anyother custom location when I start the app.

I don't want the config files to be part of the distribution tarball itself.

#### Any good examples for using sbt-native-packager

I am trying to figure out the native packager plugin for use in by sbt.

Are there any good sources / examples for me to go through apart from the one in https://github.com/sbt/sbt-native-packager

Any examples where it is used in Build.scala instead of build.sbt types?

#### Multiple project dependencies in SBT native packager

I am using the SBT native packager plugin (https://github.com/sbt/sbt-native-packager) for a project composed of multiple modules.

In my SBT settings I have:

lazy val settings = packageArchetype.java_application ++ Seq(
...
// Java is required to install this application
debianPackageDependencies in Debian ++= Seq("java2-runtime"),

// Include the module JAR in the ZIP file
mappings in Universal <+= (packageBin in Compile) map { jar =>
jar -> ("lib/" + jar.getName)
}
)


The problem is that the generated ZIP, or DEB for example, do not seem to include my project's modules dependencies. There is only the final module JAR, and the libraries used in it, but not the modules that it depends on.

Do you know how could I fix that?

#### Why does sbt-native-packager generate a hardcoded top-level parent directory in distribution packages?

For example, in Archives.scala:

def makeZip(target: File, name: String, mappings: Seq[(File, String)]): File = {
...
val m2 = mappings map { case (f, p) => f -> (name +"/"+p) }

def makeTarball(compressor: File => File, ext: String)(target: File, name: String, mappings: Seq[(File, String)]): File = {
...
val m2 = mappings map { case (f, p) => f -> (rdir / name / p) }


I'm trying to understand the reason why this additional directory "name" must be hardcoded inside these functions. In the old distribution packages (prior to sbt-native-packager), I don't believe this extra directory level was there. We're trying to use sbt-native-packager to create some distribution zip/tgz's which do not have this, and for the moment have copied and hacked the code inside Archives.scala so that the mappings: Seq[(File, String)] correspond directly to the structure inside the distribution packages rather than with having the additional "name" directory, but obviously this is not ideal. Perhaps someone familiar with this domain could shed some light on this? (@jsuereth, I'm looking at you :))

### CompsciOverflow

#### Calculating the classification threshold efficiently

I have a multi-label classification problem, in which each input sample has a set of zero or more output labels.

I have a multi-class classifier which, for every input sample, returns a certain score for each of the output labels.

One way to use this multi-class classifier for multi-label classification is to calculate a threshold score, and select for the output all labels that get a score above the threshold.

My question is: what is the most efficient way to calculate the threshold score?

Currently we use the following scheme:

• Train the multi-class classifier on 90% of the training set.
• Run the base multi-class classifier on the remaining 10% of the training set (which we call "development set"). Keep all the scores in a large array.
• Create a set of all scores that are returned by the base multi-class classifier for any input and any label. Call the set $S$.
• For every score $s \in S$, and for every input sample in the development set, calculate the positive labels returned when the threshold is $s$. Compare to the gold-standard. Calculate the precision, recall and $F1$.
• Select the score $s_max$ that yielded the maximum $F1$.

This is not efficient because there are many different scores and the method is linear in the number of different scores.

We plotted the $F1$ against the scores and noticed that this is a concave function - it has a single maximum. So, we thought of using binary search to find the maximum point. But, we are not sure that the function will always be concave.

What advice do you have for making this process more efficient?

### StackOverflow

Functional Programming in Scala shows this implementation of compose:

def compose[A, B, C](f: A => F[B], g: B => F[C]): A => F[C] = {
a => flatMap(f(a))(g)
}


How can I understand the a in front of =>?

#### Atomic getOrElseUpdate on Scala Map

I am looking for a Map implementation which supports an atomic getOrElseUpdate, which is not the case for neither Java concurrent maps implementation, nor Scala ones.

I know there was a challenge, http://boundary.com/blog/2011/05/03/the-atomicmap-challenge-solutions/ about that, and the project here that holds the solution:

However this project is not maintained (compatible only with Scala 2.9.x)

Is there any available ready-to-use implementation of such a map with atomic getOrElseUpdate?

#### slick: how to get rows without knowing number of columns

I am running a stored procedure as plain sql in scala slick. This means that number of columns are determined at runtime based on paramters passed to stored procedure. Since the schema is not known before hand, I cant map to a class.

Is there anyway to get each of the rows as a list of strings?

Tried: sql"stored_procedure @param=value".as[List[String]]
Got Error: could not find implicit value for parameter rconv: scala.slick.jdbc.GetResult[List[String]]

I am also wondering how GetResult should be implemented with variable number of columns.
implicit val getResult = GetResult(r => CantMapToClassLikethis(r.<<, r.<<))

Any help is really appreciated

#### ChaCha20 and Poly1305 in OpenSSH

OpenBSD developer Damien Miller (djm@) wrote a great post titled "ChaCha20 and Poly1305 in OpenSSH" and below is a small excerpt:

Recently, I committed support for a new authenticated encryption cipher for OpenSSH, chacha20-poly1305@openssh.com. This cipher combines two primitives from Daniel J. Bernstein: the ChaCha20 cipher and the Poly1305 MAC (Message Authentication Code) and was inspired by Adam Langley's similar proposal for TLS.

Why another cipher and MAC? A few reasons... First, we would like a high-performance cipher to replace RC4 since it is pretty close to broken now, we'd also like an authenticated encryption mode to complement AES-GCM - which is great if your hardware supports it, but takes significant voodoo to make run in constant time and, finally, having an authenticated encryption mode that is based on a stream cipher allows us to encrypt the packet lengths again.

Wait, what do you mean by "encrypt the packet lengths again"? (last rhetorical question, I promise) Well, it's a long story that requires a little background...

### StackOverflow

#### How to avoid a stack overflow with State monad

I encountered a stack overflow using scalaz State Monad.

Here is the culprit iterator.toList.traverseS_(collectOne). the iterator contains a lot of elements (10M)

• Q1 : iterator.toList force the evaluation, that I want avoid. But I need a list to call traverseS_
• Q2 : Any idea for a workaround to avoid the stack overflow ?

I think cutting the process in sublists but I have to combine the State at the end… The question is now how to transform Iterable[State] -> State

#### Getting result of a spawned function in Erlang

My objective at the moment is to write Erlang code calculating a list of N elements, where each element is a factorial of it's "index" (so, for N = 10 I would like to get [1!, 2!, 3!, ..., 10!]). What's more, I would like every element to be calculated in a seperate process (I know it is simply inefficient, but I am expected to implement it and compare its efficiency with other methods later).

In my code, I wanted to use one function as a "loop" over given N, that for N, N-1, N-2... spawns a process which calculates factorial(N) and sends the result to some "collecting" function, which packs received results into a list. I know my concept is probably overcomplicated, so hopefully the code will explain a bit more:

messageFactorial(N, listPID) ->
listPID ! factorial(N).      %% send calculated factorial to "collector".

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
nProcessesFactorialList(-1) ->
ok;
nProcessesFactorialList(N) ->
spawn(pFactorial, messageFactorial, [N, listPID]),   %%for each N spawn...
nProcessesFactorialList(N-1).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
listPrepare(List) ->            %% "collector", for the last factorial returns
receive                     %% a list of factorials (1! = 1).
1 -> List;
X ->
listPrepare([X | List])
end.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
startProcessesFactorialList(N) ->
register(listPID, spawn(pFactorial, listPrepare, [[]])),
nProcessesFactorialList(N).


I guess it shall work, by which I mean that listPrepare finally returns a list of factorials. But the problem is, I do not know how to get that list, how to get what it returned? As for now my code returns ok, as this is what nProcessesFactorialList returns at its finish. I thought about sending the List of results from listPrepare to nProcessesFactorialList in the end, but then it would also need to be a registered process, from which I wouldn't know how to recover that list.

So basically, how to get the result from a registered process running listPrepare (which is my list of factorials)? If my code is not right at all, I would ask for a suggestion of how to get it better. Thanks in advance.

#### best practice for record method in clojure

i have a record:

(defrecord Foo [a b])


and an instance method for it

(defn inc-a-field [this] (into this {:a (inc (:a this))}))


is it best practice to define a protocol for that? (since it is Foo specific)

### StackOverflow

#### Heterogeneous arguments in a Scala function

How can I pass some HList as an argument? So I can make in a such way:

def HFunc[F, S, T](hlist: F :: S :: T :: HNil) {
// here is some code
}

HFunc(HList(1, true, "String")) // it works perfect


But if I have a long list, and I dunno nothing about it, how can I make some operations on it? How can I pass argument and not to loose its type?

#### Eclipse Scala IDE: can't open hierarchy for standard library classes

I have exactly the same problem as in this question: Eclipse: Using "Open Declaration" ... in a Scala project

However, I'm using the latest Scala IDE in version 3.0.2 (I have downloaded the Eclipse bundle from the site), and I would assume such basic functionality works by now, and apparently it's me who have something misconfigured.

I have created a new Scala project. Then I open some standard library class/trait/whatever, let's say scala.util.parsing.combinator.JavaTokenParsers. The source is neatly displayed, but when I try to show class hierarchy, I get the message: The resource is not on the build path of a Java project.

Also, searching for references etc. won't work.

I guess it is a matter of properly configuring the build path? Or maybe I should somehow attach Scala library sources to my project? But I can see the source, so aren't they attached already?

Here is the snapshot of my project configuration:

UPDATE:

By playing a bit with setting/resetting build path stuff, I managed to get rid of pop-up warning but the class hierarchy comes up empty and when searching for references I get only hits from my own sources, nothing from standard library.

In another workspace I also tried randomly adding and removing scala-library jars and got it work almost, but the type hierarchy comes up only with super-classes, without any sub-classes (which renders it quite useless). Searching for references works ok though.

Funny thing, I cannot make it work in my original workspace...

Gotta love Eclipse.

#### MD5 Message Digest from SUN, <in progress>

I try to get MD5 string using Java, but the function below return string "MD5 Message Digest from SUN, <in progress>":

public String hash(String value) {
try {
MessageDigest md = MessageDigest.getInstance("MD5");
md.update(value.getBytes("UTF-8"));
return md.toString();
} catch (NoSuchAlgorithmException e) {
return null;
} catch (UnsupportedEncodingException e) {
return null;
}
}


I'm using OpenJDK on Xubuntu. Why I get this message? Is there a way to get MD5 hash using this setup?

#### How to compile SNA (Scala Native Access)

I'm new in Scala and Java world, so please to forgive me this simple question.

In my project I want to use native library, which is the interface to the hardware device. I'm trying to compile SNA (Scala Native Access), but I can't do that.

I downloaded SNA source to folder sna, and JNA jars to folder jna next cd to D:\sna\src\com\sun\jna\java and try to compile with this command:

javac -cp d:\JNA\jna-4.0.0.jar Structure.java

It gave me following errors:

Structure.java:1050: error: cannot find symbol
if (m.getSize() < requiredSize) {
^
symbol:   method getSize()
location: variable m of type Memory
Structure.java:1287: error: cannot find symbol
StructField sf = ((Union)ref).biggestField;
^
symbol:   variable biggestField
location: class Union
Note: Structure.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
2 errors


If SNA is even supported by the author? If it is compatible with actual version of JNA (4.0.0)?

I'm trying this steps on Windows7 64bit.

#### Importing anorm.SQL in IntelliJ IDEA 13

• IntelliJ 13 Community Edition
• Play Framework 2.2
• Scala 2.10.2

I am importing anorm._ and using SQL in my object. The object starts off as follows:

package controllers

import play.api.mvc._
import play.api.db.DB
import play.api.Play.current
import anorm._

object Walks extends Controller {

val futureWalksSql = SQL("SELECT * FROM walks where evt_date > now()")


IntelliJ cannot resolve symbol SQL. If I ctrl+Enter, after anorm. there is no SQL option, although there is a .Sql trait, object and class.

When I run the play project, it all works just fine, with no compilation errors so this Scala is syntactically correct but IntelliJ isn't picking this up. I have created the idea files by calling idea from within the play console and I've also tried idea with-sources=yes.

How do I get IntelliJ Community Editon to pick up anorm.SQL? What is so special about this object? I'm still learning Scala and so this might be a Scala issue.

### StackOverflow

#### Using an HLists as an arguments to a function

Is it possible to pass an HList as a function parameter?

This is what I have so far:

import shapeless._
import poly._

object id extends (Id ~> Id) {
def apply[T](f: shapeless.Id[T]): shapeless.Id[T] = f
}

object ListHandler {
def mapOverHlist[ListType <: HList](list: ListType) = list.map(id)
}

val x = "foo" :: "bar" :: 0xb33f :: HNil
ListHandler.mapOverHlist(x)


could not find implicit value for parameter mapper:
shapeless.ops.hlist.Mapper[ShapelessSpec.this.id.type,ListType]


### CompsciOverflow

#### Is there a more up-to-date / wider-scope version of the 'Compendium of NP Optimization Problems'

When I was studying Comp Sci, we had Garey & Johnson as a course textbook, with a large collection of NP-Complete problems. But by that time you could also have a look at the Compendium of NP Optimization Problems.

However, it seems that site has not seen update in several years. Is that indeed the case? Is there a more up-to-date compendium (perhaps in print?) which accounts for further research and contains more problems in more domains?

#### Finding numbers that can be expressed as a sum of cubes in two different ways [on hold]

I'm trying to write a program to find numbers that can be expressed as a sum of cubes in two different ways. Here's what I have so far (in Java):

/** Prints the first ARGS[0] numbers who can be expressed as the sum of two
*  cubes in two different ways. */
public static void main(String[] args) {
int n = Integer.parseInt(args[0]);
int count = 0;
int currVal = 1;
while (count < n) {
if (isSpecialNumber(currVal)) {
System.out.println(currVal);
count += 1;
}
currVal += 1;
}
}

/** Returns true iff N can be expressed as the sum of two cubes in two
*  different ways. */
public static boolean isSpecialNumber(int n) {
}


Any ideas on how to proceed with isSpecialNumber?

### StackOverflow

#### How can I configure system properties or logback configuration variables from typesafe config?

I have variables with defaults in my logback.xml configuration file, and I would like to be able to optionally set these variables from my typesafe config application.conf file.

I am deploying the application using one-jar, and the application.conf file packaged up in the deployable jar contains defaults. I pass -Dconfig.file=foo.conf on execution to provide the path to a server-specific config file.

Right now I can also pass -Dlog.level and other variables to override my defaults in logback.xml, and I also have to pass -Dfile.encoding=UTF-8 on the command line. I'm looking for a way to be able to specify these properties in the typesafe config instead of on the command line. It feels like there should be a way to do it, but I can't find the magic words.

logback.xml:

<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path:-logs/}/${log.file:-myLog.log}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>

<!-- keep 15 days' worth of history -->
<maxHistory>${log.history.days:-15}</maxHistory> </rollingPolicy> <encoder> <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern> </encoder> </appender> <root level="${log.level:-INFO}">
<appender-ref ref="FILE" />
</root>
</configuration>


application.conf (bundled):

akka {
log-config-on-start = false
event-handlers = ["akka.event.slf4j.Slf4jEventHandler"]
}


sample server-specific app.conf:

include "/application.conf"

akka.log-config-on-start = true

log.level = WARN // this is what I'd LIKE to be able to do


How I'm currently running the app:

java -Dfile.encoding=UTF-8 -Dconfig.file=myApp.conf -Dlog.level=WARN -jar myApp_2.10-0.1-one-jar.jar


I have a certain function that has exponential run time and should be executed in the background.

If a user does not wish to wait for the result, he should be able to cancel the computation with a button press. I don't want to keep anything from this computation.

Since I couldn't find a good way to cancel the computation, I embedded the algorithm into a class extending Thread and also have checks for the interrupt flag in the algorithm.

In my opinion this destroys the "beauty" of the algorithm since it is now only available with this Thread class.

I haven't done much with Futures or Actors yet. Is it possible to use them for this kind of problem? How would you approach it? Thanks in advance.

#### How to Delete a Specific Object From The Arrayof Objects in MongoDB Casbah

mydata is look like below:-

    {
"categories": [
{
"categoryname": "Eletronics",
"categoryId": "89sxop",
"displayname": "Eletronics",
"subcategories": [
{
"subcategoryname": "laptop",
"subcategoryId": "454",
"displayname": "Laptop"
},
{
"subcategoryname": "camera",
"subcategoryId": "sony123",
"displayname": "Camera"
}
]
}
]
}


I want to delete a specific object from a Subcategories Array

we are Trying like below code:-(This is to Delete Category)

val removingData = $pull(MongoDBObject("categories" -> MongoDBObject("categoryName" -> "Entertainment")))  this code is for removing Particular category. But I Want To Remove ONE or MORE subCategories from particular category. The subCategory(Ex:-i want to Delete camera Object from mydata) from the Electronics category Expected Output:-(after deleting the camera Object) { "categories": [ { "categoryname": "Eletronics", "categoryId": "89sxop", "displayname": "Eletronics", "subcategories": [ { "subcategoryname": "laptop", "subcategoryId": "454", "displayname": "Laptop" } ] } ] }  Best Regards GSY #### How to use Scala in IntelliJ IDEA (or: why is it so difficult to get a working IDE for Scala)? I recently gave up trying to use Scala in Eclipse (basic stuff like completion doesn't work). So now I'm trying IntelliJ. I'm not getting very far. I've been able to edit programs (within syntax highlighting and completion... yay!). But I'm unable to run even the simplest "Hello World". This was the original error: Scala signature Predef has wrong version Expected 5.0 found: 4.1 in .... scala-library.jar  But that was yesterday with IDEA 9.0.1. See below... UPDATE Today I uninstalled IntelliJ 9.0.1, and installed 9.0.2 Early Availability, with the 4/14 stable version of the Scala plug-in. Then I setup a project from scratch through the wizards: • new project from scratch • JDK is 1.6.u20 • accept the default (project) instead of global / module • accept the download of Scala 2.8.0beta1 into project's lib folder Created a new class: object hello { def main(args: Array[String]) { println("hello: " + args); } }  For my efforts, I now have a brand-new error :) Here it is: Scalac internal error: class java.lang.ClassNotFoundException [java.net.URLClassLoader$1.run(URLClassLoader.java:202), java.security.AccessController.doPrivileged(Native Method), java.net.URLClassLoader.findClass(URLClassLoader.java:190), java.lang.ClassLoader.loadClass(ClassLoader.java:307), sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301), java.lang.ClassLoader.loadClass(ClassLoader.java:248), java.lang.Class.forName0(Native Method), java.lang.Class.forName(Class.java:169), org.jetbrains.plugins.scala.compiler.rt.ScalacRunner.main(ScalacRunner.java:72)] FINAL UPDATE I uninstalled 9.0.2 EA and reinstalled 9.0.1, but this time went with the 2.7.3 version of Scala rather than the default 2.7.6, because 2.7.3 is the one shown in the screen-shots at the IntelliJ website (I guess the screen-shots prove that they actually tested this version!). Now everything works!!! #### coursera, "Functional Programming Principles in Scala", Setting up IntelliJ on OSX The "Functional Programming Principles in Scala" coursera is prepared for Eclipse. Here there is a tutorial about using IntelliJ instead of Eclipse for the course: http://www.guardian.co.uk/info/developer-blog/2012/sep/21/funtional-programming-principles-scala-setting-up-intellij However, it fails with: sbt gen-idea [error] Not a valid command: gen-idea [error] Expected '/' [error] Expected ':' [error] Not a valid key: gen-idea [error] gen-idea [error] ^  what can be the problem? ### /r/compsci #### Overview of cryptography without complexity theoretic assumptions. ### StackOverflow #### what is the clojure equivalent to the int class What is the equivalent clojure class to the type 'int' in java?  (def int-class (->> (seq (.getMethods java.util.Date)) (filter #(= "setDate" (.getName %))) first (.getParameterTypes) first))) int-class => int (= int-class int) => false  ### Wes Felter #### Ars Technica: Hotfile settles copyright case, agrees to$80 million in damages

Ars Technica: Hotfile settles copyright case, agrees to \$80 million in damages:

I didn’t notice back in August that the judge effectively ruled that the cyberlocker business model is illegal due to vicarious liability.

#### "The document seems fine if you are of the same point of view as the authors, and terrible if you are..."

“The document seems fine if you are of the same point of view as the authors, and terrible if you are not. Neither point of view is better than the other, so this review will probably not make anyone happy.”

- Paul Hoffman secures your router—or not

### StackOverflow

#### how to link to a image generated by javascript in play?

I am trying to convert the twitter bootstrap example into part of my play framework web-app.

In the give twitter-bootstrap example, I have the html contains

<img data-src="holder.js/900x500/auto/#666:#6a6a6a/text:Second slide" alt="Second slide">


My goal is convert it to part of my index.scala.html.

I tried following:

<img data-src="@routes.Assets.at("javascripts/holder.js/900x500/auto/#666:#6a6a6a/text:Second slide")" alt="Second slide">


It doesn't work. Any idea?

### StackOverflow

#### Comparison based on Clojure types

I have the following record type that I am trying to test:

(defrecord FirstOrderState [datum matrix]
State
;; implementation goes here ...
)


I am trying to branch based on the above type, but am not getting the results I need

(def state (->FirstOrderState datum matrix))

(= (type state) composer.algorithm.markov.state.FirstOrderState)
=> false


However, looking at the type of state confirms that it should match:

(type state)
=> composer.algorithm.markov.state.FirstOrderState


This seems like it should work, as a similar check results in true:

(= (type []) clojure.lang.PersistentVector)
=> true


What is it that I am missing here? Using the below hack provides a solution, but not very elegant:

(= (str (type state)) (str composer.algorithm.markov.state.FirstOrderState))
=> true


### StackOverflow

#### Add local dependency jar in SBT-Scalatra project

I am working on a sample scalatra webapp. I have created one more service jar which contains dao and service layer. I want to add this dependency in Scalatra-SBT project. How can i achieve this?

I bundled my service jar using command : SBT package which created a jar. How will i include this jar in web application? Can i add this in build.scala file? or can i copy into any Webapp folder ?

Is it possible to push this jar to local repository and pull it from there when my webapp builds?

#### Play Scala Custom Template Format

I want to use Play Templates to generate source code for a programming language.

I'd like to add support for a custom format to the template engine as per Play's documentation. But I do not understand:

1) Where to specify a new file extension? templatesTypes += ("code" -> "some.lang.LangFormat") in build.sbt?

2) how to stop Play from escaping HTML characters and adding blank lines

Has anyone got experience with Play custom template formats? If possible please provide links to examples.

### Planet Theory

#### Isomorphism Is Where It’s At

Isomorphism at the SODA 2014 conference

Ronald Read and Derek Corneil are Canadian mathematicians and computer scientists. Read earned a PhD in Mathematics from the University of London in 1959, while Corneil was one of the inaugural PhD’s in the University of Toronto’s Department of Computer Science. Read is also an accomplished musician and composer—indeed our photo comes from his entry with a sheet-music publisher in England—and here is a music-themed paper. Corneil became Chair at UT DCS and has done much liaison work with Canadian IT companies and international education. Together they wrote a survey paper in the 1977 first volume of the Journal of Graph Theory titled “The Graph Isomorphism Disease.”

Today Ken and I would like to take issue with one of the words in their title. No, not “disease,” but rather: “graph.”

The graph isomorphism (GI) problem itself has a long history, and already did in the 1970s. The computational problem is to determine whether two given adjacency matrices (or edge lists) of graphs in fact represent the same graph, and has considerable practical importance. The theoretical problem is whether GI has a polynomial-time algorithm; GI is in ${\mathsf{NP}}$ but has “intermediate” status of not being classified as either ${\mathsf{NP}}$-complete or in ${\mathsf{P}}$. In the 1970′s there seemed to be great progress toward the latter, as algorithms were obtained for graphs of bounded degree, and later, bounded multiplicity of eigenvalues of the adjacency matrices. But this ground to a halt, and GI has even resisted huge efforts recently to classify it into bounded-error quantum polynomial time (${\mathsf{BQP}}$), even though ${\mathsf{BQP}}$ contains another of the few and famous intermediate problems, factoring. The problem’s seduction and nettlesomeness led Read and Corneil to classify it as a mathematical disease.

Happily this disease has been manifesting itself in different forms that recently seem amenable to being “cured” by real progress. The list of accepted papers to the upcoming SODA 2014 conference has several instances:

• Lattice Isomorphism: a paper by Ishay Haviv and Oded Regev.

• Order-Type Isomorphism: a paper by Greg Aloupis, John Iacono, Stefan Langerman, Özgür Özkan, adding Stefanie Wuhrer.

• Robust Graph Isomorphism, an approximation concept developed by Ryan O’Donnell, John Wright, Chenggang Wu, and Yuan Zhou.

None of these directly targets GI or its main spinoff, group isomorphism. Still, the range of interesting and applicable isomorphism-type problems is expanding in ways we appreciate. But first some words about the originals.

## Graph and Group Isomorphism

Unfortunately, one of us—that is I, Dick—have spent more hours then I care to admit, working on these two isomorphism problems. Ages ago with Zeke Zalcstein—and independent from Bob Tarjan—I made the trivial but useful observation that group isomorphism can be solved in time ${n^{\log n +O(1)}}$. Here the group must be presented as a multiplication table. David Rosenbaum has shaved down the exponent by one or two factors of ${\frac{1}{2}}$ as we covered last spring, including a paper at SODA 2013. Just as we go to press—well to WordPress—Josh Grochow has posted to ECCC a paper with Youming Qiao that gives an ${n^{O(\log\log n)}}$-time algorithm for the special case of central-radical groups, that is groups whose center coincides with the maximum solvable normal subgroup. This extends work by László Babai and others from SODA 2011 and ICALP 2012, and the paper likewise gives more cases that are polynomial-time solvable.

Later I worked on graph isomorphism using what I call the “beacon set method.” This yielded strong results about random graphs, and also results about special families of graphs. The result on random graphs, for example, allowed an adversary to modify ${o(n)}$ of the edges of the graph, and still the isomorphism could be done in polynomial time with high probability.

The paper was never published—a mistake that one should avoid; publish all of your results. The paper did prove that isomorphism could be done in ${n^{O(\log n)}}$ time for symmetric graphs. Warning the first page of this paper comes up mostly blank, so scroll down to see the paper.

I—Ken writing some of these snippets from papers too—have remained relatively free of the isomorphism bug, except for formulating an open question about classifying “which ‘graph-unravelings’ are ${\mathsf{NP}}$-hard?” in an earlier post. Still, I agree with Dick about a nice quote in the SODA 2014 paper on order types:

Isomorphism is probably one of the most fundamental problems for any discrete structure.

So we consider the SODA 2014 topics in turn.

## Isomorphism For Lattices

The objects are lattices. Wait that word is used a lot in math. It could stand for according to this:

• Lattice (order), a partially ordered set with unique least upper bounds and greatest lower bounds.

• Lattice (group), a repeating arrangement of points.

• Lattice (discrete subgroup), a discrete subgroup of a topological group with finite co-volume.

• Lattice (module), a module over a ring embedded in a vector space over a field.

• Lattice graph, a graph that can be drawn within a repeating arrangement of points.

• Lattice multiplication, a multiplication algorithm suitable for hand calculation.

• Lattice model (finance), a method for evaluating stock options that divides time into discrete intervals.

The meaning used here is: a discrete subgroup which spans a real vector space. Every lattice, in this sense, has a basis and is formed by all integer linear combinations.

Haviv and Regev study the natural question: when are two lattices isomorphic? Here this means that there is a rotation that moves one to the other. This sounds pretty simple, perhaps easy, but low-dimensional intuition often does not work in high dimensions. They prove:

Theorem 1 Isomorphism of lattices can be determined in time ${n^{O(n)}}$.

They actually prove much more: they find all the possible isomorphic mappings. A very neat result.

Their algorithm “violates” a informal rule that I, Dick, have found to be quite useful. The rule is:

There are no natural decision algorithms whose running time is polynomial in ${n!}$.

Of course an algorithm that list all the even permutations on ${n}$ letters must run in this type of time—there are that many outputs. But I have usually found that algorithms that run in a factorial type running time can be improved to run in exponential time ${2^{cn}}$.

## Isomorphism For Order Types

The objects are sets of ${n}$ points in Euclidean ${d}$-dimensional space. The order type of a set of points is determined by the spatial structure of the points: it is not based on the exact distances of the points from each other. In the plane the order type of ${a,b,c}$ can be (i) clockwise, (ii) counter-clockwise, or (iii) collinear. The determinant of the matrix

$\displaystyle \begin{matrix} a_{x} & a_{y} & 1 \\ b_{x} & b_{y} & 1 \\ c_{x} & c_{y} & 1 \end{matrix}$

is positive, negative, or zero respectively.

Not surprisingly this all generalizes to any dimension. Two sets of ${n}$ points have the same order type provided there is a bijection between them that preserves the order.

The main result of Aloupis, Iacono, Langerman, Özkan, and Wuhrer is:

Theorem 2 There is an algorithm that determines the order type of ${n}$ points in ${d}$-dimensional Euclidean space in time ${n^{O(d)}}$.

One notable feature of this result is that the points need not be in general position—there is no restriction on they arrangement. This seems like an important feature and should make the result have many more potential applications. An obvious question that occurs to us is: If there is an algorithm for testing isomorphism of order types for general position in time ${T}$ does that imply anything about un-restriction points? Can the general case be done in ${O(T)}$?

We note also that this yields an answer to the question:

How do five people get together to work on one problem?

They started their work together at the 2011 Mid-Winter Workshop on Computational Geometry. So that is how we get papers with many authors. A great argument for more such workshops and meetings.

## Isomorphism For Noisy Cases

We can also consider various approximative notions for graph isomorphism. One aspect of this comes from an old paper I wrote in 1999 with Anna Gál, Shai Halevi, and Erez Petrank, titled “Computing From Partial Solutions.” We showed that computing a partial isomorphism—that is getting just ${\log(n)}$-many entries pairs correct that extend to some isomorphism ${\phi}$ between two graphs ${G}$ and ${H}$—is as hard as GI itself. This also represented only a partial solution to the five-author question, since we stopped at four. However, we were soon out-done by a three-author solution: André Große, Jörg Rothe, and Gerd Wechsung showed that getting even one pair ${\phi(u) = v}$ correct is as hard as computing some ${\phi}$.

The SODA 2014 paper by O’Donnell, Wright, Wu, and Zhou is emceed by Konstantin Makarychev in this CMU video, Zhou presenting. That doesn’t quite make five people. Well the paper leans on a hypothesis by Uriel Feige that any polynomial-time randomized algorithm giving one-sided error on random ${\mathsf{3XOR}\text{-}\mathsf{SAT}}$ instances—that is with clauses of the form ${(x_i \oplus x_j \oplus x_k)}$ (possibly negated). Feige’s hypothesis is that if such an algorithm never says “no” on an instance for which a ${(1-\epsilon)}$ portion of clauses can be satisfied, then it will incorrectly say “yes”” on a large fraction of instances, those for which a portion almost ${1/8}$ cannot be satisfied; this is related to the Unique Games conjecture. So it almost has a fifth author.

Given two graphs ${G}$ and ${H}$ with the same numbers ${n}$ of vertices and ${m}$ of edges, call them “almost isomorphic”” if for suitable ${\epsilon > 0}$ there is a bijection ${\phi: V(G) \longrightarrow V(H)}$ such that for all but ${\epsilon m}$ edges ${(u,v)}$ of ${G}$, ${(\phi(u),\phi(v))}$ is an edge of ${H}$. For suitable functions ${r}$ such that ${r(\epsilon) \longrightarrow 0}$ as ${\epsilon \longrightarrow 0}$, they define the “Robust GI Problem” as:

Given two graphs that are ${\epsilon}$-almost isomorphic, compute a ${\phi'}$ that preserves a ${1 - r(\epsilon)}$ portion of edges.

They show that assuming Feige’s hypothesis, there really is no polynomial-time algorithm for this problem. Or put another way, finding a polynomial-time algorithm for this problem suffices to falsify Feige’s hypothesis. This is surprising insofar as GI is not ${\mathsf{NP}}$-complete unless the polynomial hierarchy collapses, while Feige’s hypothesis involves a real form of ${\mathsf{SAT}}$. They also show unconditionally that a particular general family of polynomial-time algorithms based on semi-definite programming fails on this problem.

## Open Problems

Are there more good cases of isomorphism to study? Are there even more relationships to Unique Games? Will SODA 2015 have more than three papers on this topic?

### QuantOverflow

#### Different Quant Finance Careers question [on hold]

I am currently a junior mathematics and finance double major interested in quantitative finance, however I am not sure as to the specific type of career I want. I am very attracted to mathematics and for that reason would love a career in a quantitative field working on topics dealing with advanced mathematics. I no doubt want to go into finance but given there are so many types of quant careers, I am confused as to what a quant does. I have done some research but descriptions seem vague. Most careers I come across are: Quantitative trader and Quantitative analyst, Financial Engineering. Some of my questions are:

2.) What is financial engineering really? What does it mean to be a financial engineer? What are some mathematical problems faced by these professionals?

3.) Hedge Funds really interest me the most, what sort of careers are available at hedge funds and what type of advanced math do these professionals employ on a day to day basis?

### StackOverflow

#### Clojure not nil check

In Clojure nil? checks for nil. How does one check for not nil?

I want to do the Clojure equivalent of the following Java code:

if (value1==null && value2!=null) {
}


Follow-up: I was hoping for a not nil check instead of wrapping it with not. if has a if-not counterpart. Is there such a counterpart for nil??

#### Can you call zmq_recv with a NULL buffer and 0 length to skip a message?

I am using zmq (3.2) and am utilizing multipart messages. I am handling some error cases where the first message part is invalid and I want to discard the remaining parts. What I was thinking was something along the lines of.

void zmq_drain_multipart(void *sock)
{
int more;
size_t size = sizeof(int);
zmq_getsockopt(sock, ZMQ_RCVMORE, &more, &size);
while (more)
{
zmq_recv(sock, NULL, 0, 0);
zmq_getsockopt(sock, ZMQ_RCVMORE, &more, &size);
}
}


Is something like this supposed to work.

#### Predicate function instance? in Clojure not behaving as expected

I have a user-defined record type that I am trying to check against using instance?, however I don't quite understand why it is evaluating in the way that it is.

I have the following record

(defrecord FirstOrderState [pitch matrix]
State
;; implementation here ...
)


I have a function that is branching based on the type or class of the predicate, however, it appears to always evaluate to false when I expect it to evaluate to true, i.e. at the REPL:

(def state (->FirstOrderState pitch matrix))
=> #'user/state

(type state)
=> composer.algorithm.markov.state.FirstOrderState

(class state)
=> composer.algorithm.markov.state.FirstOrderState


However, if I do either of the following

(instance? composer.algorithm.state.markov.FirstOrderState (class state))
=> false

(instance? composer.algorithm.state.markov.FirstOrderState (type state))
=> false


instance evaluates to false when I expect it to evaluate to true`. What am I missing here? Is there something special about records that won't allow you to make a check in this way, and if so, is there an alternative?