willnill
V2EX  ›  问与答

求助 Spark 问题,弄了一天了!!!!

  •  
  •   willnill · Apr 9, 2018 · 1596 views
    This topic created in 2983 days ago, the information mentioned may be changed or developed.
    现有 dataDF 结构如下:
    root
    |-- asin: string (nullable = true)
    |-- topic: array (nullable = true)
    | |-- element: double (containsNull = true)

    | t1 | t2 | t3 |
    a1 | d11 | d12 | d13 |
    a2 | d21 | d22 | d23 |
    a3 | d31 | d32 | d33 |

    请问怎么用 Statistics.corr 方法来计算两两的相似度,从而得到如下结构的 DF:
    case class Corr(asin1: String, asin2: String, corr: Double)
    No Comments Yet
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   4052 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 37ms · UTC 05:14 · PVG 13:14 · LAX 22:14 · JFK 01:14
    ♥ Do have faith in what you're doing.