scalacenter/scaladex

The Scala Package Index

scaladex's Issues

group scalaJs crossDeps

UI: Contribution policy badges

• When clicked, should take you to project's CONTRIBUTING.md file (if it exists)

https://github.com/blog/1184-contributing-guidelines

Search page

• pagination + style
• order by (stars, fork, latest release)
• keywords (api)

Hand-tagging popular projects with keywords

Templating engine

Clean up search (no RCs/milestones/etc)

Better aggregation of projects. E.g., there should only ever be one akka/akka-http found in the search, not many.

notify new release via gitter.im

Internal error when querying API

http://localhost:8080/api/find?query=rapture
generates There was an internal server error.
with Console stacktrace

webappJVM [ERROR] [05/31/2016 01:13:36.036] [scaladex-akka.actor.default-dispatcher-31] [akka.actor.ActorSystemImpl(scaladex)] Error during processing of request HttpRequest(HttpMethod(GET),http://localhost:8080/api/find?query=rapture,List(Host: localhost:8080, Connection: keep-alive, Accept: text/html, application/xhtml+xml, application/xml;q=0.9, image/webp, */*;q=0.8, Upgrade-Insecure-Requests: 1, User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36, Accept-Encoding: gzip, deflate, sdch, Accept-Language: en-US, en;q=0.8, fr;q=0.6, Timeout-Access: <function1>),HttpEntity.Strict(none/none,ByteString()),HttpProtocol(HTTP/1.1))
webappJVM upickle.Invalid$Data: Key Missing: preRelease (data: {
webappJVM     "major": "1",
webappJVM     "minor": "1",
webappJVM     "patch": [
webappJVM         "0"
webappJVM     ]
webappJVM })
webappJVM   at upickle.GeneratedUtil$class.mapToArray(GeneratedUtil.scala:45)
webappJVM   at upickle.default$.mapToArray(Api.scala:25)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:90)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:89)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Types$class.readJs(Types.scala:160)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.Generated$$anonfun$Tuple4R$1.applyOrElse(Generated.scala:39)
webappJVM   at upickle.Generated$$anonfun$Tuple4R$1.applyOrElse(Generated.scala:39)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:90)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:89)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Types$class.readJs(Types.scala:160)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.Generated$$anonfun$Tuple9R$1.applyOrElse(Generated.scala:79)
webappJVM   at upickle.Generated$$anonfun$Tuple9R$1.applyOrElse(Generated.scala:79)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:90)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:89)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Types$Reader$$anon$2.applyOrElse(Types.scala:103)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Types$class.readJs(Types.scala:160)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1$$anonfun$applyOrElse$1.apply(Implicits.scala:146)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1$$anonfun$applyOrElse$1.apply(Implicits.scala:146)
webappJVM   at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
webappJVM   at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
webappJVM   at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
webappJVM   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
webappJVM   at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
webappJVM   at scala.collection.AbstractTraversable.map(Traversable.scala:104)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1.applyOrElse(Implicits.scala:146)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1.applyOrElse(Implicits.scala:146)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Types$class.readJs(Types.scala:160)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.Generated$$anonfun$Tuple3R$1.applyOrElse(Generated.scala:31)
webappJVM   at upickle.Generated$$anonfun$Tuple3R$1.applyOrElse(Generated.scala:31)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:90)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:89)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Types$Reader$$anon$2.applyOrElse(Types.scala:103)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Types$class.readJs(Types.scala:160)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1$$anonfun$applyOrElse$1.apply(Implicits.scala:146)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1$$anonfun$applyOrElse$1.apply(Implicits.scala:146)
webappJVM   at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
webappJVM   at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
webappJVM   at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
webappJVM   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
webappJVM   at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
webappJVM   at scala.collection.AbstractTraversable.map(Traversable.scala:104)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1.applyOrElse(Implicits.scala:146)
webappJVM   at upickle.Implicits$$anonfun$SeqishR$1.applyOrElse(Implicits.scala:146)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Types$class.readJs(Types.scala:160)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.Generated$$anonfun$Tuple8R$1.applyOrElse(Generated.scala:71)
webappJVM   at upickle.Generated$$anonfun$Tuple8R$1.applyOrElse(Generated.scala:71)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:90)
webappJVM   at upickle.Implicits$$anonfun$CaseR$1.applyOrElse(Implicits.scala:89)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Types$Reader$$anon$2.applyOrElse(Types.scala:103)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:53)
webappJVM   at upickle.Implicits$Internal$$anon$1.apply(Implicits.scala:50)
webappJVM   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
webappJVM   at upickle.Implicits$Internal$$anon$1.applyOrElse(Implicits.scala:50)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:113)
webappJVM   at upickle.Types$Reader$$anon$2.apply(Types.scala:103)
webappJVM   at upickle.Types$class.readJs(Types.scala:160)
webappJVM   at upickle.default$.readJs(Api.scala:25)
webappJVM   at upickle.Types$class.read(Types.scala:156)
webappJVM   at upickle.default$.read(Api.scala:25)
webappJVM   at ch.epfl.scala.index.data.ProjectProtocol$ProjectAs$.as(package.scala:15)
webappJVM   at ch.epfl.scala.index.data.ProjectProtocol$ProjectAs$.as(package.scala:13)
webappJVM   at com.sksamuel.elastic4s.RichSearchHit.as(RichSearchResponse.scala:99)
webappJVM   at com.sksamuel.elastic4s.RichSearchResponse$$anonfun$as$1.apply(RichSearchResponse.scala:32)
webappJVM   at com.sksamuel.elastic4s.RichSearchResponse$$anonfun$as$1.apply(RichSearchResponse.scala:32)
webappJVM   at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
webappJVM   at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
webappJVM   at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
webappJVM   at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
webappJVM   at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
webappJVM   at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
webappJVM   at com.sksamuel.elastic4s.RichSearchResponse.as(RichSearchResponse.scala:32)
webappJVM   at ch.epfl.scala.index.server.ApiImplementation.ch$epfl$scala$index$server$ApiImplementation$$$anonfun$2(ApiImplementation.scala:40)
webappJVM   at ch.epfl.scala.index.server.ApiImplementation$lambda$$find$1.apply(ApiImplementation.scala:35)
webappJVM   at ch.epfl.scala.index.server.ApiImplementation$lambda$$find$1.apply(ApiImplementation.scala:35)
webappJVM   at scala.util.Success$$anonfun$map$1.apply(Try.scala:237)
webappJVM   at scala.util.Try$.apply(Try.scala:192)
webappJVM   at scala.util.Success.map(Try.scala:237)
webappJVM   at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)
webappJVM   at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)
webappJVM   at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
webappJVM   at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
webappJVM   at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
webappJVM   at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
webappJVM   at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
webappJVM   at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
webappJVM   at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
webappJVM   at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
webappJVM   at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:405)
webappJVM   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
webappJVM   at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
webappJVM   at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
webappJVM   at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Dependencies & Reverse Dependencies

part of the #34 refactor

Edit Project

After being logged in a user can edit his project info:

accessible fields:

• Project
• Artifact
• default artifact #158
• #100 scaladoc link

UI

• dropdown multiselect for keywords
• input box for other infos
• dropdown for artifact deprecation

Backend

• keywords dropdown api
• deprecation dropdown api
• update api

UI: Display all project versions on project page

@MasseGuillaume will show how to get all project versions from backend for use by front-end

http -> https for scala-lang.org

Search bar, add project description to drop down

@MasseGuillaume will make sure cleaner search results are returned and that the description field is available. But it would be nice to include the project description in the drop-down search bar à la npm https://www.npmjs.com

UI: badge per license type, hover-over explaining it

UI: Login via GitHub

UI side of things – Heather will further clarify what we mean in this ticket

Scala Days Berlin v2

• ui for oauth2 login (#12)
• collaborator can edit information (#49)
• improve search relevance (#56)
• new projects, new releases in front page (#54)
• select artifact & release + target in project page (#58)
• keywords in project page (#21)
• + license info (#22)
• contributor wanted (#25)
• cookie warning (#3)
• repo statistics from GitHub visible (#115)
• restore autocomplete/dropdown bar (#122)

Search relevance

Index private repositories

Eugene Platonov at ebay said it would be helpful to index private repos. For example, they could look for reverse dependencies to notify them for breaking api changes.

Deploy to Alaska

cookie warning!

https://silktide.com/tools/cookie-consent/docs/

opensearch.xml

http://www.chromium.org/tab-to-search

filter artifact & releases in project page

two dropdowns:

• artifact name
• release

filter on backend

Integrate look & feel with scala-lang.org

make html links in Github Readme absolute

API

JSON APIs for accessing Scaladex data

• Joris's needs

In the future: (to keep in mind)

• Felix will potentially need for linking dottydoc
• Adriaan for community build

Main Scaladex landing page

Partial Update of Data

• created_after https://bintray.com/docs/api/#_file_search_by_name (6f01f12)
• update elasticsearch without loosing information

Github Oauth2

Connect front page with Backend

api to receive new pom

duplicate #27

Admin-level users, edit-all capabilities

Backend/UI both. Use already written Oauth2 stuff to autheneticate a list of "admins". As low-level as possible for now.

follow redirects when downloading github readmes

https://github.com/scalacenter/scaladex2/blob/master/data/src/main/scala/ch.epfl.scala.index.data/github/GithubDownload.scala#L73

this should be a stream graph with retries

Get dependencies on EPFL machine

Subdomain of scala-lang for package index

Possible names:

• index.scala-lang.org
• package.scala-lang.org

render app server side

important libs not showing up

Fetch more info from Github

• Definition: "Lightweight, modular, and extensible library for functional programming."
• Website: "http://typelevel.org/cats/"
• Stars/Fork: 1,075/24
• Readme
• Contributors (+Avatar, etc)

sort releases by version

For scala-pickling for example, the list of version numbers is out of order:

The current version should be v.0.10.1. As-is, the order in the above list doesn't make a whole lot of sense.

UI: add keywords to UI

sbt publish should update/notify scalaindex

@raulraja: contribute to sbt to make sure publish notify the Scala Index

We could use various social vectors such as mailing list, slack,gitter, irc, twitter to broadcast a library release.

Example for twitter:

@scala_index: @shapeless 2.3.1 is released http://index.scala-lang.org/typelevel/shapeless

Some pages can't fetch README

An example is here: https://index.scala-lang.org/lihaoyi/derive

Also, this page is repeating the right column again.

Nightly job to update index

Receiving requests from sbt publish tool

Add keywords to backend. Searchable

nginx config

servername: index.scala-lang.org
proxy https => application

UI: Updating Project Info

• Which fields will be updatable? (@MasseGuillaume + @heathermiller will provide a list)
• @MasseGuillaume makes sure there's some update function to be used by the front-end

Way for companies to endorse projects

notify new release via twitter

Input: New releases arrive via two different channels:

• pulling from bintray's API every day
• receving a push from maven central as soon as they have a new release

Output: We want to notify Twitter about when we have a new release.

• We can push to a central Twitter account.
• We can let the project owner customize a tweet template:

For example:

@playframework just released <artifact-name> <version> for <Scala target> 
<link to scaladex>

Index scala-lang.org poms

Use Jenkins to build/deploy

Build + deploy upon Github post-commit