diff options
| author | Ellen Spertus <ellen.spertus@gmail.com> | 2018-08-13 10:27:48 -0700 |
|---|---|---|
| committer | Ellen Spertus <ellen.spertus@gmail.com> | 2018-08-13 10:27:48 -0700 |
| commit | b4f1acce5eccbb56291f82906d9c01534c7f1506 (patch) | |
| tree | 96ff33ed95a4eb9304280b1d5f1ccb269c0d0424 /scalding/src/main/scala/sandcrawler/GrobidScorable.scala | |
| parent | 1c6e1234974d8b6e4480a13ff5c4ff861c6d1deb (diff) | |
| download | sandcrawler-b4f1acce5eccbb56291f82906d9c01534c7f1506.tar.gz sandcrawler-b4f1acce5eccbb56291f82906d9c01534c7f1506.zip | |
Factored out ScorableFeatures.
Diffstat (limited to 'scalding/src/main/scala/sandcrawler/GrobidScorable.scala')
| -rw-r--r-- | scalding/src/main/scala/sandcrawler/GrobidScorable.scala | 6 |
1 files changed, 1 insertions, 5 deletions
diff --git a/scalding/src/main/scala/sandcrawler/GrobidScorable.scala b/scalding/src/main/scala/sandcrawler/GrobidScorable.scala index 94b3494..5ba7d58 100644 --- a/scalding/src/main/scala/sandcrawler/GrobidScorable.scala +++ b/scalding/src/main/scala/sandcrawler/GrobidScorable.scala @@ -35,11 +35,7 @@ object GrobidScorable { case None => MapFeatures(Scorable.NoSlug, json) case Some(map) => { if (map contains "title") { - val map2 = Scorable.toScorableMap(Scorable.getString(map, "title"), - sha1=key) - new MapFeatures( - Scorable.mapToSlug(map2), - JSONObject(map2).toString) + new ScorableFeatures(Scorable.getString(map, "title"), sha1=key).toMapFeatures } else { MapFeatures(Scorable.NoSlug, json) } |